Channel Attention Block Research Articles

Attention-guided aggregation stereo matching network

Existing stereo matching networks based on deep learning lack multi-level and multi-module attention and integration for feature information. Therefore, we propose an attention-guided aggregation stereo matching network to encode and integrate information multiple times. Specifically, we design a residual network based on the 2D channel attention block to adaptively calibrate weight response, improving the robustness of the feature representation. We also construct a 3D stacked hourglass structure based on the 3D channel attention block to calibrate the weight response of the 4D cost volume in the channel dimension, further enhancing the network guidance and aggregation capabilities. In addition, we introduce a 4D guided cost volume, which pre-groups the extracted image features and exploits the similarity measures in each group to guide the concatenation features, further realizing interactive learning of cost volume. The experimental results on the Scene Flow and KITTI benchmark datasets showed that the proposed network significantly improves the prediction disparity accuracy with a small increase in calculation time.

Scale channel attention network for image segmentation

The object scale variation results in a negative effect on image segmentation performance. Spatial pyramid pooling module or the attention mechanism are two widely used components in deep neural networks to handle this problem. Applying the single component commonly achieves limited benefit. To push the limit, in this paper, we propose a scale channel attention network (SCA-Net), which enhances the fusion feature of multi-scale by using channel attention components. After the multiple-scale pooling step, the multi-scale spatial information distributes in different feature channels. Meanwhile, the channel attention block is employed to guide SCA-Net focus on the object-relevant scale channels. We further explore the channel attention block and find a simple yet effective structure to combine global average pooling and global maximum pooling, resulting in a robust global information encoder. The SCA-Net does not contain any time-consuming post-processing, which is an extra step after the neural network for the segmentation result optimization. The assessment results on PASCAL VOC 2012 and Cityscapes benchmarks achieve the test set performance of 75.5% and 77.0%.

Adaptive weighted crowd receptive field network for crowd counting

Crowd counting plays an important role in crowd analysis and monitoring. To this end, we propose a novel method called Adaptive Weighted Crowd Receptive Field Network (AWRFN) for crowd counting to estimate the number of people and the spatial distribution of input crowd images. The proposed AWRFN is composed of four modules: backbone, crowd receptive field block (CRFB), recurrent block (RB), and channel attention block (CAB). Backbone utilizes the first ten layers of VGG16 to extract base features of input images. CRFB is a multi-branch architecture simulating a real human visual system for further obtaining refined and discriminative crowd features. RB generates strong semantic and global information by recurrently stacking convolutional layers with the same parameters. CAB outputs appropriate weights to supervise each channel of the feature maps output from CRFB, which uses the outputs of RB as guidance. Different from previous works using Euclidean Loss, we employ L1_Smooth Loss to train our network in an end-to-end fashion. To demonstrate the effectiveness of our proposed method, we implement AWRFN on two representative datasets including the ShanghaiTech dataset and the UCF_CC_50 dataset. The experimental results prove that our method is both effective and robust compared with the state-of-the-art approaches.

Image dehazing with uneven illumination prior by dense residual channel attention network

Existing dehazing methods based on convolutional neural networks estimate the transmission map by treating channel-wise features equally, which lacks flexibility in handling different types of haze information, leading to the poor representational ability of the network. Besides, the scene lights are predicted by an even illumination prior which does not work for a real situation. To solve these problems, the authors propose a dense residual channel attention network (DRCAN) for estimating the transmission map and use an image segmentation strategy to predict scene lights. Specifically, DRCAN is built based on the proposed dense residual block (DRB) and dense residual channel attention block (DRCAB). DRB extracts the hierarchical features with increasing receptive fields. DRCAB makes the network focus on the features containing heavy haze information. After the transmission map is estimated, fuzzy partition entropy combined with graph cuts is used to segment the transmission map into scene regions covered with varying scene lights. This strategy not only considers the fuzzy intensities of the low-contrast transmission map but also takes spatial correlation into account. Finally, a clear image is obtained by the transmission map and varying scene lights. Extensive experiments demonstrate that our method is comparable to most of existing methods.

AttPNet: Attention-Based Deep Neural Network for 3D Point Set Analysis.

Point set is a major type of 3D structure representation format characterized by its data availability and compactness. Most former deep learning-based point set models pay equal attention to different point set regions and channels, thus having limited ability in focusing on small regions and specific channels that are important for characterizing the object of interest. In this paper, we introduce a novel model named Attention-based Point Network (AttPNet). It uses attention mechanism for both global feature masking and channel weighting to focus on characteristic regions and channels. There are two branches in our model. The first branch calculates an attention mask for every point. The second branch uses convolution layers to abstract global features from point sets, where channel attention block is adapted to focus on important channels. Evaluations on the ModelNet40 benchmark dataset show that our model outperforms the existing best model in classification tasks by 0.7% without voting. In addition, experiments on augmented data demonstrate that our model is robust to rotational perturbations and missing points. We also design a Electron Cryo-Tomography (ECT) point cloud dataset and further demonstrate our model’s ability in dealing with fine-grained structures on the ECT dataset.

Adaptive Residual Channel Attention Network for Single Image Super-Resolution

Single image super-resolution (SISR) is a traditional image restoration problem. Given an image with low resolution (LR), the task of SISR is to find the homologous high-resolution (HR) image. As an ill-posed problem, there are works for SISR problem from different points of view. Recently, deep learning has shown its amazing performance in different image processing tasks. There are works for image super-resolution based on convolutional neural network (CNN). In this paper, we propose an adaptive residual channel attention network for image super-resolution. We first analyze the limitation of residual connection structure and propose an adaptive design for suitable feature fusion. Besides the adaptive connection, channel attention is proposed to adjust the importance distribution among different channels. A novel adaptive residual channel attention block (ARCB) is proposed in this paper with channel attention and adaptive connection. Then, a simple but effective upscale block design is proposed for different scales. We build our adaptive residual channel attention network (ARCN) with proposed ARCBs and upscale block. Experimental results show that our network could not only achieve better PSNR/SSIM performances on several testing benchmarks but also recover structural textures more effectively.

Multibranch Spatial-Channel Attention for Semantic Labeling of Very High-Resolution Remote Sensing Images

Very high-resolution (VHR) remote sensing images can provide fine but sometimes trivial ground object details; thus, the semantic labeling of VHR images is a challenging task. To improve the VHR labeling performance, spatial multiscale information and channel attention have been employed recently. However, the exploitation of global object features is still limited, which leads to the loss of capturing within-class variation from location to location. In this letter, we present a multibranch spatial-channel attention (MSCA) model to efficiently extract global dependency and combine it with multiscale and channel attention methods. In the spatial multiscale attention block, a multibranch feature fusion model is established to exploit the global relationship captured by self-attention and the multiscale correlation learned from dilated convolutions. To alleviate the computational cost of pixel-by-pixel self-attention operation, a spatial pyramid compressing method is also designed. In the channel attention block, average and max global pooling strategies are applied, respectively, in two channel attention branches to generalize global information from different perspectives. Those two blocks are then adaptively united by learnable weighting parameters. Experiments on two VHR image data sets demonstrate that the proposed network can yield better performance in comparison with state-of-the-art labeling methods tested.

A Robust Fabric Defect Detection Method Based on Improved RefineDet.

This paper proposes a robust fabric defect detection method, based on the improved RefineDet. This is done using the strong object localization ability and good generalization of the object detection model. Firstly, the method uses RefineDet as the base model, inheriting the advantages of the two-stage and one-stage detectors and can efficiently and quickly detect defect objects. Secondly, we design an improved head structure based on the Full Convolutional Channel Attention (FCCA) block and the Bottom-up Path Augmentation Transfer Connection Block (BA-TCB), which can improve the defect localization accuracy of the method. Finally, the proposed method applies many general optimization methods, such as attention mechanism, DIoU-NMS, and cosine annealing scheduler, and verifies the effectiveness of these optimization methods in the fabric defect localization task. Experimental results show that the proposed method is suitable for the defect detection of fabric images with unpattern background, regular patterns, and irregular patterns.

A strong feature representation for siamese network tracker

Object tracking has important application in assistive technologies for personalized monitoring. Recent trackers choosing AlexNet as their backbone to extract features have gained great success. However, AlexNet is too shallow to form a strong feature representation, the tracker based on the Siamese network have an accuracy gap compared with state-of-the-art algorithms. To solve this problem, this paper proposes a tracker called SiamPF. Firstly, the modified pre-trained VGG16 network is fine-tuned as the backbone. Secondly, an AlexNet-like branch is added after the third convolutional layer and merged with the response map of the backbone network to form a preliminary strong feature representation. And then, a channel attention block is designed to adaptively select the contribution features. Finally, the APCE is modified to process the response map to reduce interference and focus the tracker on the target. Our SiamPF only used ILSVRC2015-VID for training, but it achieved excellent performance on OTB-2013 / OTB-2015 / VOT2015 / VOT2017, while maintaining the real-time performance of 41FPS on the GTX 1080Ti.

Channel and Spatial Attention Regression Network for Cup-to-Disc Ratio Estimation

Cup-to-disc ratio (CDR) is of great importance during assessing structural changes at the optic nerve head (ONH) and diagnosis of glaucoma. While most efforts have been put on acquiring the CDR number through CNN-based segmentation algorithms followed by the calculation of CDR, these methods usually only focus on the features in the convolution kernel, which is, after all, the operation of the local region, ignoring the contribution of rich global features (such as distant pixels) to the current features. In this paper, a new end-to-end channel and spatial attention regression deep learning network is proposed to deduces CDR number from the regression perspective and combine the self-attention mechanism with the regression network. Our network consists of four modules: the feature extraction module to extract deep features expressing the complicated pattern of optic disc (OD) and optic cup (OC), the attention module including the channel attention block (CAB) and the spatial attention block (SAB) to improve feature representation by aggregating long-range contextual information, the regression module to deduce CDR number directly, and the segmentation-auxiliary module to focus the model’s attention on the relevant features instead of the background region. Especially, the CAB selects relatively important feature maps in channel dimension, shifting the emphasis on the OD and OC region; meanwhile, the SAB learns the discriminative ability of feature representation at pixel level by capturing the relationship of intra-feature map. The experimental results of ORIGA dataset show that our method obtains absolute CDR error of 0.067 and the Pearson’s correlation coefficient of 0.694 in estimating CDR and our method has a great potential in predicting the CDR number.

A part-based attention network for person re-identification

Person re-identification (re-id) is the task of recognizing images of the same pedestrian captured by different cameras with non-overlapping views. Person re-id is a challenging task due to the existence of large view variations, such as spatial misalignment, background clutter and human poses change. In this paper, we handle these challenges from the following two aspects: utilizing attention mechanism to alleviate misalignment problem and exploiting the complementary effects of global-local features for more stable pedestrian descriptors. Specifically, we first present a part-based attention model consisting of a channel attention block and a spatial attention block to sequentially refine the convolutional descriptors of person body parts. The channel and spatial attention blocks weight the channels and positions of body-part feature maps to spot the informative channels and regions, respectively. Then global full-body and local body-part of the refined feature maps are pooled into global and local representations, which are jointly trained using identity classification loss. We conduct extensive experiments on four standard benchmark datasets including Market1501, CUHK03, DukeMTMC-reID, and CUHK01, and the experimental results demonstrate the effectiveness of the presented method.

Pyramid Attention Aggregation Network for Semantic Segmentation of Surgical Instruments

Semantic segmentation of surgical instruments plays a critical role in computer-assisted surgery. However, specular reflection and scale variation of instruments are likely to occur in the surgical environment, undesirably altering visual features of instruments, such as color and shape. These issues make semantic segmentation of surgical instruments more challenging. In this paper, a novel network, Pyramid Attention Aggregation Network, is proposed to aggregate multi-scale attentive features for surgical instruments. It contains two critical modules: Double Attention Module and Pyramid Upsampling Module. Specifically, the Double Attention Module includes two attention blocks (i.e., position attention block and channel attention block), which model semantic dependencies between positions and channels by capturing joint semantic information and global contexts, respectively. The attentive features generated by the Double Attention Module can distinguish target regions, contributing to solving the specular reflection issue. Moreover, the Pyramid Upsampling Module extracts local details and global contexts by aggregating multi-scale attentive features. It learns the shape and size features of surgical instruments in different receptive fields and thus addresses the scale variation issue. The proposed network achieves state-of-the-art performance on various datasets. It achieves a new record of 97.10% mean IOU on Cata7. Besides, it comes first in the MICCAI EndoVis Challenge 2017 with 9.90% increase on mean IOU.

Accurate single image super-resolution using multi-path wide-activated residual network

In many recent image super-resolution (SR) methods based on convolutional neural networks (CNNs), the superior performance was achieved by training very large networks, which may not be suitable for real-world applications with limited computing resources. Therefore, it is necessary to develop more compact networks that achieve a better trade-off between the model size and the performance. In this paper, we propose an efficient and effective network called multi-path wide-activated residual network (MWRN). Firstly, as the basic building block of MWRN, the multi-path wide-activated residual block (MWRB) is presented to extract the multi-scale features. MWRB consists of three parallel wide-activated residual paths, where the dilated convolutions with different dilation factors are used to increase the receptive fields. Secondly, the fusional channel attention (FCA) module, which contains a bottleneck layer and a multi-path wide-activated residual channel attention (MWRCA) block, is designed to well exploit the multi-level features in MWRN. In each FCA, the MWRCA block refines the fused features by taking the interdependencies among feature channels into consideration. The experiments demonstrate that, compared with the state-of-the-art methods, the proposed MWRN model is able to provide very competitive performance with a relatively small number of parameters.

Classification of Hyperspectral Image Based on Double-Branch Dual-Attention Mechanism Network

In recent years, researchers have paid increasing attention on hyperspectral image (HSI) classification using deep learning methods. To improve the accuracy and reduce the training samples, we propose a double-branch dual-attention mechanism network (DBDA) for HSI classification in this paper. Two branches are designed in DBDA to capture plenty of spectral and spatial features contained in HSI. Furthermore, a channel attention block and a spatial attention block are applied to these two branches respectively, which enables DBDA to refine and optimize the extracted feature maps. A series of experiments on four hyperspectral datasets show that the proposed framework has superior performance to the state-of-the-art algorithm, especially when the training samples are signally lacking.

MSD-Net: Multi-Scale Discriminative Network for COVID-19 Lung Infection Segmentation on CT.

Since the first patient reported in December 2019, 2019 novel coronavirus disease (COVID-19) has become global pandemic with more than 10 million total confirmed cases and 500 thousand related deaths. Using deep learning methods to quickly identify COVID-19 and accurately segment the infected area can help control the outbreak and assist in treatment. Computed tomography (CT) as a fast and easy clinical method, it is suitable for assisting in diagnosis and treatment of COVID-19. According to clinical manifestations, COVID-19 lung infection areas can be divided into three categories: ground-glass opacities, interstitial infiltrates and consolidation. We proposed a multi-scale discriminative network (MSD-Net) for multi-class segmentation of COVID-19 lung infection on CT. In the MSD-Net, we proposed pyramid convolution block (PCB), channel attention block (CAB) and residual refinement block (RRB). The PCB can increase the receptive field by using different numbers and different sizes of kernels, which strengthened the ability to segment the infected areas of different sizes. The CAB was used to fusion the input of the two stages and focus features on the area to be segmented. The role of RRB was to refine the feature maps. Experimental results showed that the dice similarity coefficient (DSC) of the three infection categories were 0.7422,0.7384,0.8769 respectively. For sensitivity and specificity, the results of three infection categories were (0.8593, 0.9742), (0.8268,0.9869) and (0.8645,0.9889) respectively. The experimental results demonstrated that the network proposed in this paper can effectively segment the COVID-19 infection on CT images. It can be adopted for assisting in diagnosis and treatment of COVID-19.

CA-GAN: Class-Condition Attention GAN for Underwater Image Enhancement

Underwater images suffer from serious color distortion and detail loss because of the wavelength-dependent light absorption and scattering, which seriously influences the subsequent underwater object detection and recognition. The latest methods for underwater image enhancement are based on deep models, which focus on finding a mapping function from the underwater image subspace to a ground-truth image subspace. They neglect the diversity of underwater conditions which leads to different background colors of underwater images. In this paper, we propose a Class-condition Attention Generative Adversarial Network (CA-GAN) to enhance an underwater image. We build an underwater image dataset which contains ten categories generated by the simulator with different water attenuation coefficient and depth. Relying on the underwater image classes, CA-GAN creates a many-to-one mapping function for an underwater image. Moreover, in order to generate the realistic image, attention mechanism is utilized. In the channel attention block, the feature maps in the front-end layers and the back-end layers are fused along channels, and in the spatial attention block, feature maps are pixel-wise fused. Extensive experiments are conducted on synthetic and real underwater images. The experimental results demonstrate that CA-GAN can effectively recover color and detail of various scenes of underwater images and is superior to the state-of-the-art methods.

Model-Based Deep Network for Single Image Deraining

For current learning-based single image deraining methods, deraining networks are usually designed based on a simplified linear additive rain model, which may not only cause unreal synthetic rainy images for both training and testing datasets, but also adversely affect the applicability and generality of corresponding networks. In this paper, we use the screen blend model of Photoshop as the nonlinear rainy image decomposition model. Based on this model, we design a novel channel attention U-DenseNet for rain detection and a residual dense block for rain removal. The detection sub-network not only adjusts channel-wise feature responses by our novel channel attention block to pay more attention to learn the rain map, but also combines the context information with the precise localization by the U-DenseNet to promote pixel-wise estimation accuracy. After rain detection, we use the nonlinear model to get a coarse rain-free image, and then introduce a deraining refinement subnetwork consisted of the residual dense block to obtain a fine rain-free image. For training our network, we apply the nonlinear rain model to synthesize a benchmark dataset called as RITD. It contains 3200 triplets of rainy images, rain maps, and clean background images. Our extensive quantitative and qualitative experimental results show that our method outperforms several state-of-the-art methods on both synthetic and real images.

Exploiting Raw Images for Real-Scene Super-Resolution.

Super-resolution is a fundamental problem in computer vision which aims to overcome the spatial limitation of camera sensors. While significant progress has been made in single image super-resolution, most algorithms only perform well on synthetic data, which limits their applications in real scenarios. In this paper, we study the problem of real-scene single image super-resolution to bridge the gap between synthetic data and real captured images. We focus on two issues of existing super-resolution algorithms: lack of realistic training data and insufficient utilization of visual information obtained from cameras. To address the first issue, we propose a method to generate more realistic training data by mimicking the imaging process of digital cameras. For the second issue, we develop a two-branch convolutional neural network to exploit the radiance information originally-recorded in raw images. In addition, we propose a dense channel-attention block for better image restoration as well as a learning-based guided filter network for effective color correction. Our model is able to generalize to different cameras without deliberately training on images from specific camera types. Extensive experiments demonstrate that the proposed algorithm can recover fine details and clear structures, and achieve high-quality results for single image super-resolution in real scenes.

Real-Time Dense Semantic Labeling with Dual-Path Framework for High-Resolution Remote Sensing Image

Dense semantic labeling plays a pivotal role in high-resolution remote sensing image research. It provides pixel-level classification which is crucial in land cover mapping and urban planning. With the recent success of the convolutional neural network (CNN), accuracy has been greatly improved by previous works. However, most networks boost performance by involving too many parameters and computational overheads, which results in more inference time and hardware resources, while some attempts with light-weight networks do not achieve satisfactory results due to the insufficient feature extraction ability. In this work, we propose an efficient light-weight CNN based on dual-path architecture to address this issue. Our model utilizes three convolution layers as the spatial path to enhance the extraction of spatial information. Meanwhile, we develop the context path with the multi-fiber network (MFNet) followed by the pyramid pooling module (PPM) to obtain a sufficient receptive field. On top of these two paths, we adopt the channel attention block to refine the features from the context path and apply a feature fusion module to combine spatial information with context information. Moreover, a weighted cascade loss function is employed to enhance the learning procedure. With all these components, the performance can be significantly improved. Experiments on the Potsdam and Vaihingen datasets demonstrate that our network performs better than other light-weight networks, even some classic networks. Compared to the state-of-the-art U-Net, our model achieves higher accuracy on the two datasets with 2.5 times less network parameters and 22 times less computational floating point operations (FLOPs).

Attention-based convolutional neural network for deep face recognition

Discriminative feature embedding is of essential importance in the field of large scale face recognition. In this paper, we propose an attention-based convolutional neural network (ACNN) for discriminative face feature embedding, which aims to decrease the information redundancy among channels and focus on the most informative components of spatial feature maps. More specifically, the proposed attention module consists of a channel attention block and a spatial attention block which adaptively aggregate the feature maps in both channel and spatial domains to learn the inter-channel relationship matrix and the inter-spatial relationship matrix, then matrix multiplications are conducted for a refined and robust face feature. With the attention module we proposed, we can make standard convolutional neural networks (CNNs), such as ResNet-50, ResNet-101 have more discriminative power for deep face recognition. The experiments on Labelled Faces in the Wild (LFW), Age Database (AgeDB), Celebrities in Frontal Profile (CFP) and MegaFace Challenge 1 (MF1) show that our proposed ACNN architecture consistently outperforms naive CNNs and achieves the state-of-the-art performance.