Multi-scale Context Aggregation Research Articles

Enhancing instance segmentation: Leveraging multiscale feature fusion and attention mechanisms for automated fish weight estimation

The accurate estimation of fish weight relies on the crucial parameter of individual fish contour features. While instance segmentation proves effective in extracting fish contours, challenges arise from diverse fish postures and reduced image sharpness underwater. Current instance segmentation methods often struggle to effectively balance global and local detailed features, which can result in inaccurate positioning of contour keypoints and consequently limit the accuracy of fish weight estimation. To overcome this, our study introduces a novel instance segmentation network tailored for precise fish contour extraction. The proposed approach incorporates multi-scale feature fusion and an attention mechanism based on the Segmenting Objects by Locations (SOLO) network, referred to as SOLO-MFFA. This paper designs a multi-scale context aggregation module to integrate features with a wider range of receptive fields, augmenting the model's capability to comprehend both local features and global information. At the same time, the introduction of a mixed domain attention mechanism emphasizes more critical channel features, and simultaneously improves the localization accuracy of contour points. Compared with SOLO and its improved model CAM-SOLO on the fish instance segmentation dataset, SOLO-MFFA’s demonstrated an effective improvement, with a 4.3% and 1.6% increase in mAP (mean Average Precision), respectively. The Decoupled-SOLO-MFFA achieved higher mAP. The visualization results also demonstrate that the contour features extracted in this paper are smoother and more accurately positioned. Additionally, in comparison to other well-known instance segmentation networks, our method has demonstrated significant improvements in both qualitative and quantitative evaluations. Furthermore, the integration of contour features derived from Decoupled-SOLO-MFFA, along with binocular vision, was utilized for the precise estimation of fish perimeter and weight. The findings reveal a strong correlation between the perimeter calculated by Decoupled-SOLO-MFFA and the actual weight, with a notably reduced error in weight estimation. Compared to previous methods, RMSE, MAE, and MAPE of the linear model constructed in this paper decreased by 3.92, 3.19, and 1.4%.

PGKD-Net: Prior-guided and Knowledge Diffusive Network for Choroid Segmentation

The thickness of the choroid is considered to be an important indicator of clinical diagnosis. Therefore, accurate choroid segmentation in retinal OCT images is crucial for monitoring various ophthalmic diseases. However, this is still challenging due to the blurry boundaries and interference from other lesions. To address these issues, we propose a novel prior-guided and knowledge diffusive network (PGKD-Net) to fully utilize retinal structural information to highlight choroidal region features and boost segmentation performance. Specifically, it is composed of two parts: a Prior-mask Guided Network (PG-Net) for coarse segmentation and a Knowledge Diffusive Network (KD-Net) for fine segmentation. In addition, we design two novel feature enhancement modules, Multi-Scale Context Aggregation (MSCA) and Multi-Level Feature Fusion (MLFF). The MSCA module captures the long-distance dependencies between features from different receptive fields and improves the model’s ability to learn global context. The MLFF module integrates the cascaded context knowledge learned from PG-Net to benefit fine-level segmentation. Comprehensive experiments are conducted to evaluate the performance of the proposed PGKD-Net. Experimental results show that our proposed method achieves superior segmentation accuracy over other state-of-the-art methods. Our code is made up publicly available at: https://github.com/yzh-hdu/choroid-segmentation.

MAEU‐NET: A novel supervised architecture for brain tumor segmentation

AbstractA brain tumor is an abnormal growth of cells that damages the neural system and may lead to severe conditions. These cells are irregular in shape and size, and as a result, the manual analysis of these cells from magnetic resonance (MR) images is a cumbersome and time‐consuming task. To alleviate these issues, a novel U‐Net inspired MAEU‐Net (Multiscale context Aggregation and Edge activation U‐Net) is implemented to detect the brain tumor pixels from MR images more precisely. The encoder block of the proposed model is improved by integrating with multiscale feature aggregation and convolution block attention modules (CBAM) enabling the model to extract both pertinent low‐level and high‐level features. Further, the extracted features are subjected to a parallel pooling module to elevate the features that extract the tumor pixels. In addition to this, the output features of the last encoder are fed to a parallel dilated context aggregation module to mitigate the hyper‐parameter size and further elevate the receptive field strength of the proposed model. Apart from this, edge‐enhanced modules are incorporated with skip connections which concatenate the feature maps of the decoder with respective edge‐enhanced encoder feature maps. As a result, this module increases the aspect ratio and reduces blurry edge problems. To illustrate the performance of the proposed MAEU‐Net, it is trained and tested using Kaggle LGG, BraTS 2018, and BraTS 2021 publicly available datasets. The proposed model outperformed the previously existing models by achieving the test loss, dice coefficient, sensitivity, and specificity of 0.0590, 0.901, 0.942, and 0.998, respectively. Simultaneously, the proposed model is quantitatively analyzed by performing ablation and comparative studies and presented that the proposed MAEU‐Net is superior to other previous existing models like U‐Net++, SegNet, etc.

Attention-guided multi-scale context aggregation network for multi-modal brain glioma segmentation.

Accurate segmentation of brain glioma is a critical prerequisite for clinical diagnosis, surgical planning and treatment evaluation. In current clinical workflow, physicians typically perform delineation of brain tumor subregions slice-by-slice, which is more susceptible to variabilities in raters and also time-consuming. Besides, even though convolutional neural networks (CNNs) are driving progress, the performance of standard models still have some room for furtherimprovement. To deal with these issues, this paper proposes an attention-guided multi-scale context aggregation network (AMCA-Net) for the accurate segmentation of brain glioma in the magnetic resonance imaging (MRI) images withmulti-modalities. AMCA-Net extracts the multi-scale features from the MRI images and fuses the extracted discriminative features via a self-attention mechanism for brain glioma segmentation. The extraction is performed via a series of down-sampling, convolution layers, and the global context information guidance (GCIG) modules are developed to fuse the features extracted for contextual features. At the end of the down-sampling, a multi-scale fusion (MSF) module is designed to exploit and combine all the extracted multi-scale features. Each of the GCIG and MSF modules contain a channel attention (CA) module that can adaptively calibrate feature responses and emphasize the most relevant features. Finally, multiple predictions with different resolutions are fused through different weightings given by a multi-resolution adaptation (MRA) module instead of the use of averaging or max-pooling to improve the final segmentationresults. Datasets used in this paper are publicly accessible, that is, the Multimodal Brain Tumor Segmentation Challenges 2018 (BraTS2018) and 2019 (BraTS2019). BraTS2018 contains 285 patient cases and BraTS2019 contains 335 cases. Simulations show that the AMCA-Net has better or comparable performance against that of the other state-of-the-art models. In terms of the Dice score and Hausdorff 95 for the BraTS2018 dataset, 90.4% and 10.2 mm for the whole tumor region (WT), 83.9% and 7.4 mm for the tumor core region (TC), 80.2% and 4.3 mm for the enhancing tumor region (ET), whereas the Dice score and Hausdorff 95 for the BraTS2019 dataset, 91.0% and 10.7 mm for the WT, 84.2% and 8.4 mm for the TC, 80.1% and 4.8 mm for the ET. The proposed AMCA-Net performs comparably well in comparison to several state-of-the-art neural net models in identifying the areas involving the peritumoral edema, enhancing tumor, and necrotic and non-enhancing tumor core of brain glioma, which has great potential for clinical practice. In future research, we will further explore the feasibility of applying AMCA-Net to other similar segmentationtasks.

DBE-Net: Dual Boundary-Guided Attention Exploration Network for Polyp Segmentation

Automatic segmentation of polyps during colonoscopy can help doctors accurately find the polyp area and remove abnormal tissues in time to reduce the possibility of polyps transforming into cancer. However, the current polyp segmentation research still has the following problems: blurry polyp boundaries, multi-scale adaptability of polyps, and close resemblances between polyps and nearby normal tissues. To tackle these issues, this paper proposes a dual boundary-guided attention exploration network (DBE-Net) for polyp segmentation. Firstly, we propose a dual boundary-guided attention exploration module to solve the boundary-blurring problem. This module uses a coarse-to-fine strategy to progressively approximate the real polyp boundary. Secondly, a multi-scale context aggregation enhancement module is introduced to accommodate the multi-scale variation of polyps. Finally, we propose a low-level detail enhancement module, which can extract more low-level details and promote the performance of the overall network. Extensive experiments on five polyp segmentation benchmark datasets show that our method achieves superior performance and stronger generalization ability than state-of-the-art methods. Especially for CVC-ColonDB and ETIS, two challenging datasets among the five datasets, our method achieves excellent results of 82.4% and 80.6% in terms of mDice (mean dice similarity coefficient) and improves by 5.1% and 5.9% compared to the state-of-the-art methods.

A Multi-Scale Edge Constraint Network for the Fine Extraction of Buildings from Remote Sensing Images

Building extraction based on remote sensing images has been widely used in many industries. However, state-of-the-art methods produce an incomplete segmentation of buildings owing to unstable multi-scale context aggregation and a lack of consideration of semantic boundaries, ultimately resulting in large uncertainties in predictions at building boundaries. In this study, efficient fine building extraction methods were explored, which demonstrated that the rational use of edge features can significantly improve building recognition performance. Herein, a fine building extraction network based on a multi-scale edge constraint (MEC-Net) was proposed, which integrates the multi-scale feature fusion advantages of UNet++ and fuses edge features with other learnable multi-scale features to achieve the effect of prior constraints. Attention was paid to the alleviation of noise interference in the edge features. At the data level, according to the improvement of copy-paste according to the characteristics of remote sensing imaging, a data augmentation method for buildings (build-building) was proposed, which increased the number and diversity of positive samples by simulating the construction of buildings to increase the generalization of MEC-Net. MEC-Net achieved 91.13%, 81.05% and 74.13% IoU on the WHU, Massachusetts and Inria datasets, and it has a good inference efficiency. The experimental results show that MEC-Net outperforms the state-of-the-art methods, demonstrating its superiority. MEC-Net improves the accuracy of building boundaries by rationally using previous edge features.

MCA-Unet: A multiscale context aggregation U-Net for the segmentation of COVID-19 lesions from CT images

The pandemic of coronavirus disease (COVID-19) caused the world to face an existential health crisis. COVID-19 lesions segmentation from CT images is nowadays an essential step to assess the severity of the disease and the amount of damage to the lungs. Deep learning has brought about a breakthrough in medical image segmentation where U-Net is the most prominent deep network. However, in this study, we argue that its architecture still lacks in certain aspects. First, there is an incompatibility in the U-Net skip connection between the encoder and decoder features which adversely affects the final prediction. Second, it lacks capturing multiscale context information and ignores the contribution of all semantic information through the segmentation process. Therefore, we propose a model named MCA-Unet, a novel multiscale deep learning segmentation model, which proposes some modifications to improve upon the U-Net model. MCA-Unet is integrated with a multiscale context aggregation module which is constituted of two blocks; a context embedding block (CEB) and a cascaded dilated convolution block (CDCB). The CEB aims at reducing the semantic gap between the concatenated features along the U-Net skip connections, it enriches the low-level encoder features with rich semantics inherited from the subsequent higher-level features, to reduce the semantic gap between the low-processed encoder features and the highly-processed decoder features, thus ensuring effectual concatenation. The CDCB is integrated to address the variability in shape and size of the COVID-19 lesions, it captures global context information by gradually expanding the receptive field, then operates reversely to capture the small fine details that might be scattered by enlarging the receptive field. To validate the robustness of our model, we tested it on a publicly available dataset of 1705 axial CT images with different types of COVID-19 infection. Experimental results show that MCA-Unet has attained a remarkable gain in performance in comparison with the basic U-Net and its variant. It achieved high performance using different evaluation metrics showing 88.6% Dice similarity coefficient, 85.4% Jaccard index, and 93.5% F-score measure. This outperformance shows great potential to help physicians during their examination and improve the clinical workflow.

AMCA: Attention-Guided Multiscale Context Aggregation Network for Remote Sensing Image Change Detection

Remote sensing image change detection is the key to understanding surface changes. Although the existing change detection methods have achieved good results, some structural details are missing and the detection accuracy needs to be improved. Therefore, we propose an attention-guided multi-scale context aggregation network (AMCA) for remote sensing image change detection. First, we use the fully attentional pyramid module (FAPM) to enhance the deep feature information of the original image. And we introduce the dense feature fusion module (DFFM) to fully fuse the bi-temporal features to obtain the change regions. Second, the introduction of channel-wise cross fusion transformer (CCT) and channel-wise cross attention (CCA) not only can effectively fuse channel features focusing on different semantic patterns, but also bridge the semantic gap between multi-scale features. Next, we use the transformer decoder to map the learned high-level semantic information into the pixel space to refine the original features. In addition, we use the context extraction module (CEM) to obtain the local and global associations of feature maps. Finally, the addition of attention aggregation module (AAM) can effectively combine the feature information at different scales. Extensive experiments on three public change detection datasets show that the proposed method has advantages over other methods in terms of both visual interpretation and quantitative analysis.

MISNet: Multiscale Cross-Layer Interactive and Similarity Refinement Network for Scene Parsing of Aerial Images

Although progress has been made in multisource data scene parsing of natural scene images, extracting complex backgrounds from aerial images of various types and presenting the image at different scales remain challenging. Various factors in high-resolution aerial images (HRAIs), such as imaging blur, background clutter, object shadow, and high resolution, substantially reduce the integrity and accuracy of object segmentation. By applying multisource data fusion, as in scene parsing of natural scene images, we can solve the aforementioned problems through the integration of auxiliary data into HRAIs. To this end, we propose a multiscale cross-layer interactive and similarity refinement network (MISNet) for scene parsing of HRAIs. First, in a feature fusion optimization module, we extract, filter, and optimize multisource features and further guide and optimize the features using a feature guidance module. Second, a multiscale context aggregation module increases the receptive field, captures semantic information, and extracts rich multiscale background features. Third, a dense decoding module fuses the global guidance information and high-level fused features. We also propose a joint learning method based on feature similarity and a joint learning module to obtain deep multilevel information, enhance feature generation, and fuse multiscale and global features to enhance network representation for accurate scene parsing of HRAIs. Comprehensive experiments on two benchmark HRAIs datasets indicate that our proposed MISNet is qualitatively and quantitatively superior to similar state-of-the-art models.

HookNet: Efficient Multiscale Context Aggregation for High-Accuracy Detection of Fabric Defects

Automatic defect detection is a critical stage of quality control in the textile industry. Because defects vary in texture structure, size, and spatial distribution, existing defect detection methods have difficulty in achieving a perfect trade-off between detection efficiency, accuracy, and generalizability. This paper presents an advanced fabric defect detection model based on a deep learning algorithm. A novel parallel dilated attention module was designed: this is nested in the deep layers of the neural network backbone to establish global channel dependencies and capture multiscale contextual information. Further, a hook-shaped feature pyramid network was developed for multiscale context aggregation: this directs the network to focus on lower-level features and is helpful for further improving detection efficiency without sacrificing detection accuracy. In addition, the Alpha-GIoU loss is used to improve the accuracy of bounding box regression because of its modulated power parameter α. The advantages and effectiveness of the proposed method were statistically analyzed using data from several public datasets. The code is available at https://github.com/aabb605/HookNet.

BFMNet: Bilateral feature fusion network with multi-scale context aggregation for real-time semantic segmentation

As a key technology for scene understanding, real-time semantic segmentation is an important topic in the field of computer vision in recent years. However, some lightweight networks designed for real-time semantic segmentation only have limited receptive field, so they cannot effectively perceive multi-scale objects in images. In addition, although the fast downsampling feature extraction network reduces the amount of computation, it has the problem of loss of detailed information, resulting in poor prediction accuracy. In this paper, we propose an efficient lightweight semantic segmentation network called BFMNet to address these issues. First, we use a lightweight bilateral structure to encode the semantic and detailed information from images respectively and introduce feature interactions during the encoding process. Furthermore, we design a novel Multi-Scale Context Aggregation Module (MSCAM) to help the network perceive the information of multi-scale objects, which is crucial for semantic segmentation. Finally, we introduce a new fusion module (AEFM) that uses attention perception to facilitate bilateral feature fusion. Our network achieves competitive results on three popular semantic segmentation benchmarks: Cityscapes, CamVid and COCO-Stuff. Specifically, on a single 2080Ti GPU, our network yields 77.7% mIoU at 63.7 FPS on Cityscapes test set with the input resolution of 768×1536. Considering the speed-accuracy trade-off, we also report the results with 1024×2048 input resolution: 78.9% mIoU at 31.4 FPS. On the Camvid test set, our network achieves 75.6% mIoU at 95.8 FPS, while on the COCO-Stuff validation set, our network achieves 31.2% mIoU.

Recurrent context-aware multi-stage network for single image deraining

Single image rain streak removal is extremely necessary since rainy images can seriously affect many computer vision systems. In this paper, we propose a novel recurrent context-aware multi-stage network (ReCMN) for image rain removal that gradually predicts clean derained results. Specifically, the ReCMN introduces a multi-stage strategy to perform contextual relationship modeling. Firstly, we use the densely residual extraction block (DREB) to guide feature extraction. Then, a multi-scale context aggregation block (MCAB) is designed to utilize the long-distance dependencies and multiple scale features, which can fuse features of different levels to fully exploit contextual information. Finally, we develop a parallel attention block (PAB) to capture the channel and spatial information and only pass effective feature representation. Experimental results demonstrate that our method outperforms several state-of-the-art methods, based on both synthetic datasets and real-world rainy images.

Improvement of noisy images filtered by bilateral process using a multi-scale context aggregation network

Deep learning has recently received a lot of attention as a feasible solution to a variety of artificial intelligence difficulties. Convolutional neural networks (CNNs) outperform other deep learning architectures in the application of object identification and recognition when compared to other machine learning methods. Speech recognition, pattern analysis, and image identification, all benefit from deep neural networks. When performing image operations on noisy images, such as fog removal or low light enhancement, image processing methods such as filtering or image enhancement are required. The study shows the effect of using Multi-scale deep learning Context Aggregation Network CAN on Bilateral Filtering Approximation (BFA) for de-noising noisy CCTV images. Data-store is used tomanage our dataset, which is an object or collection of data that are huge to enter in memory, it allows to read, manage, and process data located in multiple files as a single entity. The CAN architecture provides integral deep learning layers such as input, convolution, back normalization, and Leaky ReLu layers to construct multi-scale. It is also possible to add custom layers like adaptor normalization (µ) and adaptive normalization (Lambda) to the network. The performance of the developed CAN approximation operator on the bilateral filtering noisy image is proven when improving both the noisy reference image and a CCTV foggy image. The three image evaluation metrics (SSIM, NIQE, and PSNR) evaluate the developed CAN approximation visually and quantitatively when comparing the created de-noised image over the reference image.Compared with the input noisy image, these evaluation metrics for the developed CAN de-noised image were (0.92673/0.76253, 6.18105/12.1865, and 26.786/20.3254) respectively

Hairlines removal and low contrast enhancement of melanoma skin images using convolutional neural network with aggregation of contextual information

Skin cancer image preprocessing in dermoscopic images is a fundamental yet challenging task in designing highly efficient computerized diagnostic systems. The presence of hairlines and poor resolution of the skin images poses a significant problem in accurately identifying features for automatic cancer classification. Two approaches are presented here; the first is intensity adjustment-based hair removal (IA-HR) to occlude hairlines by using morphological operators to adjust intensity levels and using nearest neighbour criteria to detect and eliminate hair pixels. A multiscale context aggregation convolutional neural network (MCACNN) is used in the second approach to reduce high-frequency content loss and increase the resolution of dermoscopy images. Evaluation of the proposed methods was conducted using the publicly available dataset HAM10000. For comparison, an additional benchmark dataset H13-Sim was used comprising 13 dermoscopic skin cancer images. Experimental results showed that the proposed methods outperformed state-of-the-art methods using PSNR, SSIM, NIQE, and BRISQUE metrics. It is concluded that the IA-HR method can occlude hairlines effectively while the MCACNN model can improve the resolution of skin cancer images. Consequently, the proposed methods may be used as part of a preprocessing phase in developing a diagnostic system.

A CNN-Transformer Network With Multiscale Context Aggregation for Fine-Grained Cropland Change Detection

Non-agriculturalization incidents are serious threats to local agricultural ecosystem and global food security. Remote sensing change detection (CD) can provide an effective approach for in-time detection and prevention of such incidents. However, existing CD methods are difficult to deal with the large intra-class differences of cropland changes in high-resolution images (HRIs). In addition, traditional CNN based models are plagued by the loss of long-range context information, and the high computational complexity brought by deep layers. Therefore, we propose a CNN-transformer network with multi-scale context aggregation (MSCANet), which combines the merits of CNN and transformer to fulfill efficient and effective cropland change detection. In the MSCANet, a CNN-based feature extractor is first utilized to capture hierarchical features, then a transformer-based multi-scale context aggregator (MSCA) is designed to encode and aggregate context information. Finally, a multi-branch prediction head (MBPH) with three CNN classifiers is applied to obtain change maps, to enhance the supervision for deep layers. Besides, for the lack of change detection dataset with fine-grained cropland change of interest, we also provide a new cropland change detection dataset (CLCD), which contains 600 pairs of 512×512 bi-temporal images with the spatial resolution of 0.5-2m. Comparative experiments with several CD models prove the effectiveness of the MSCANet, with the highest F1 of 64.67% on the high-resolution semantic change detection dataset (HRSCD), and of 71.29% on CLCD. Code and dataset in the paper will be available for download from the following link https://github.com/liumency/CropLand-CD.

Joint Variation Learning of Fusion and Difference Features for Change Detection in Remote Sensing Images

Remote sensing (RS) image change detection (CD) is an earth observation technique for detecting surface changes in the same area during a period. With the rapid development of deep learning, various deep neural networks especially Siamese ones have been widely used in the field of CD. However, they have the deficiency of insufficient contextual information aggregation, resulting in false and missed detections, and it is difficult to refine the detection of change edges. To alleviate these problems and obtain more accurate results, we propose an efficient self-weighted spatial-temporal attention network (SSANet). In contrast to the Siamese structure, our network is a novel joint learning framework composed of fusion sub-network, difference sub-network, and decoder. Fusion sub-network is used to extract multiscale object features where we propose a multi-core channel-aligning attention (MCA) module to capture the long-range semantic information for multi-scale context aggregation. Difference sub-network is used to extract the difference variation features, where we propose a feature differential reconfiguration (FDR) module to learn the temporal change information. FDR can effectively filter change information and reconstruct features to improve the perception of changed regions. To better balance the MCA and FDR modules, an asymmetric weighting (AW) module is proposed in the decoder to self-weight the multi-scale features and generate the change map. Experiments demonstrate the efficiency of proposed sub-networks and modules, and the state-of-the-art performance of SSANet.

Multiscale Context Aggregation Network for Building Change Detection Using High Resolution Remote Sensing Images

The existing methods of building change detection (CD) using remote sensing (RS) images are still deficient in handling scale variation and class imbalance problems, indicating a decrease in the robustness of small-object detection and pseudo-change information. Thus, a novel building CD framework called the multiscale context aggregation network (MSCANet) is proposed. The high-resolution network is integrated into the feature extracting stage to maintain high-resolution representations throughout the whole process. Then, multiscale context information is aggregated using a scale-aware feature pyramid module (FPM). Recognition performance can be improved from discriminant feature representation learning by using a channel–spatial attention module. Furthermore, a class-balanced loss is proposed to reduce the impact of class imbalance in long-tail datasets. Experimental results from using the LEVIR-CD and SZTAKI AirChange benchmark datasets prove the superiority of the MSCANet over the other baseline methods, with improved maximum F1 scores of 5.28 and 8.47, respectively.

Medical image translation using an edge-guided generative adversarial network with global-to-local feature fusion.

In this paper, we propose a framework based deep learning for medical image translation using paired and unpaired training data. Initially, a deep neural network with an encoder-decoder structure is proposed for image-to-image translation using paired training data. A multi-scale context aggregation approach is then used to extract various features from different levels of encoding, which are used during the corresponding network decoding stage. At this point, we further propose an edge-guided generative adversarial network for image-to-image translation based on unpaired training data. An edge constraint loss function is used to improve network performance in tissue boundaries. To analyze framework performance, we conducted five different medical image translation tasks. The assessment demonstrates that the proposed deep learning framework brings significant improvement beyond state-of-the-arts.

Distillation Remote Sensing Object Counting via Multi-Scale Context Feature Aggregation

Remote sensing object counting is an important issue in remote sensing analysis. Remote sensing object counting has many challenges, such as large-scale variations and complex backgrounds. The previous counting methods have many shortboards, such as only focusing on local appearance features of target scenes and ignoring the self-supervision ability of the network itself. To remedy the above problems, in this article, we propose a novel remote sensing object counting method, which contains the adaptive multi-scale context aggregation module (AMCAM) and the self-context distillation module (SCDM). The AMCAM can model and fuse context information from different receptive fields effectively. It also keeps detailed information through multiple pixel attention (PA) modules step by step. The SCDM can improve the representation learning without adding any additional supervision information. SCDM uses feature maps from the deeper layer of the network to supervise feature maps from the earlier layer of the network. Our method has achieved good performance on the remote sensing object counting dataset, RSOC, and mainstream crowd counting datasets, such as ShanghaiTech and UCF-QNRF datasets.

Multilevel Deformable Attention-Aggregated Networks for Change Detection in Bitemporal Remote Sensing Imagery

Deep learning (DL) approaches based on convolutional encoder–decoder networks have shown promising results in bitemporal change detection. However, their performance is limited by insufficient contextual information aggregation because they cannot fully capture the implicit contextual dependency relationships among feature maps at different levels. Moreover, harvesting long-range contextual information typically incurs high computational complexity. To circumvent these challenges, we propose multilevel deformable attention-aggregated networks (MLDANets) to effectively learn long-range dependencies across multiple levels of bitemporal convolutional features for multiscale context aggregation. Specifically, a multilevel change-aware deformable attention (MCDA) module consisting of linear projections with learnable parameters is built based on multihead self-attention (SA) with a deformable sampling strategy. It is applied in the skip connections of an encoder–decoder network taking a bitemporal deep feature hypersequence (BDFH) as input. MCDA can progressively address a set of informative sampling locations in multilevel feature maps for each query element in the BDFH. Simultaneously, MCDA learns to characterize beneficial information from different spatial and feature subspaces of BDFH using multiple attention heads for change perception. As a result, contextual dependencies across multiple levels of bitemporal feature maps can be adaptively aggregated via attention weights to generate multilevel discriminative change-aware representations. Experiments on very-high-resolution (VHR) datasets verify that MLDANets outperform state-of-the-art change detection approaches with dramatically faster training convergence and high computational efficiency.