Multi-scale Contextual Information Research Articles

Reconstructing Images Through Multimode Fibers From the Up-Conversion Speckle Patterns via Deep Learning

The mode mixing and mode dispersion in the multimode fiber (MMF) will produce complex speckle patterns in the distal end of the fiber as an object passes through the MMF, rendering image reconstruction to be a challenging task. In recent years, convolutional neural networks have been successfully applied to image reconstruction from speckles. However, the imaging spectra of these studies are mostly in the visible spectrum range and require complete speckle information for reconstruction. In this paper, researchers build an optical imaging system that employs up-conversion imaging technology to collect speckle patterns generated by infrared light transmitting through a multimode fiber and a frequency-doubling crystal. They propose a speckle restoration network (SRNet) based on a generative adversarial network (GAN) to reconstruct speckle images. The generator of GAN uses ResNest and atrous spatial pyramid pooling (ASPP) to extract multi-level features and multi-scale context information, respectively. The discriminator of GAN significantly improves the quality of the reconstructed image generated by the generator. In addition, researchers adopt a special training method named pre-training generator to avoid gradient disappearance or gradient explosion in the training process. With the designed network, high-quality images were successfully reconstructed even with only a portion of the speckle information.

An Improved SSD-Like Deep Network-Based Object Detection Method for Indoor Scenes

The indoor scene object detection technology is of important research significance, which is one of the popular research topics in the field of scene understanding for indoor robots. In recent years, the solutions based on deep learning have achieved good results in object detection. However, there are still some problems to be further studied in indoor object detection methods, such as lighting problem and occlusion problem caused by the complexity of the indoor environment. Aiming at these problems, an improved object detection method based on deep neural networks is proposed in this paper, which uses a framework similar to the Single Shot MultiBox Detector (SSD). In the proposed method, an improved ResNet50 network is used to enhance the transmission of information, and the feature expression capability of the feature extraction network is improved. At the same time, a multi-scale contextual information extraction (MCIE) module is used to extract the contextual information of the indoor scene, so as to improve the indoor object detection effect. In addition, an improved dual-threshold non-maximum suppression (DT-NMS) algorithm is used to alleviate the occlusion problem in indoor scenes. Finally, the public dataset SUN2012 is further screened for the special application of indoor scene object detection, and the proposed method is tested on this dataset. The experimental results show that the mean average precision ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">mAP</i> ) of the proposed method can reach 54.10%, which is higher than those of the state-of-the-art methods.

Enhancing Multiscale Representations With Transformer for Remote Sensing Image Semantic Segmentation

Semantic segmentation is an extremely challenging task in high-resolution remote sensing (HRRS) images as objects have complex spatial layouts and enormous variations in appearance. Convolutional neural networks (CNNs) have excellent ability to extract local features and have been widely applied as the feature extractor for various vision tasks. However, due to the inherent inductive bias of convolution operation, CNNs inevitably have limitations in modeling long-range dependencies. Transformer can capture global representations well, but unfortunately ignores the details of local features and has high computational and spatial complexity in processing high-resolution feature maps. In this paper, we propose a novel hybrid architecture for HRRS image segmentation, termed EMRT, to exploit the advantages of convolution operations and Transformer to enhance multi-scale representation learning. We incorporate the deformable self-attention mechanism in the Transformer to automatically adjust the receptive field, and design an encoder-decoder architecture accordingly to achieve efficient context modeling. Specifically, the CNN is constructed to extract feature representations. In the encoder, local features and global representations at different resolutions are extracted by the CNN and Transformer, respectively, and fused in an interactive manner. Moreover, a separate spatial branch is designed to extract multi-scale contextual information as queries, and global dependencies between features at different scales are efficiently established by the decoder. Extensive experiments on three public remote sensing datasets demonstrate the superiority of EMRT and indicate that the overall performance of our method outperforms state-of-the-art methods. Code is available at https://github.com/peach-xiao/EMRT.

Exploiting Multi-Scale Contextual Information for Face Anti-Spoofing

Exploiting Multi-Scale Contextual Information for Face Anti-Spoofing

ConvSegNet: Automated Polyp Segmentation From Colonoscopy Using Context Feature Refinement With Multiple Convolutional Kernel Sizes

Colorectal cancer occurs in the rectal of humans, and early detection has been proved to reduce its mortality rate. Colonoscopy is the standard used in detecting the presence of polyps in the rectal, and accurate segmentation of the polyps from colonoscopy images often provides helpful information for early diagnosis and treatment. Although existing deep learning models often achieve high segmentation performance when tested on the same dataset used in model training; still, their performance often degrades when applied to out-of-distribution datasets, leading to low model generalization or overfitting. This challenge is often associated with the quality of the features learnt from the input images. In this work, a novel Context Feature Refinement (CFR) module is proposed to address the challenge of low model generalization and segmentation performance. The CFR module is built to extract contextual information from the incoming feature map by using multiple parallel convolutional layers with progressively increasing kernel sizes. Using multiple parallel convolutions with different kernel sizes helped to extract more efficient multi-scale contextual information and thus enabled the network to effectively identify and segment small and fine details, as well as larger and more complex structures in the input images. Extensive experiments on three public benchmark datasets in CVC-ClinicDB, Kvasir-SEG, and BKAI-NeoPolyp showed that the proposed ConvSegNet model achieved jaccard, dice and F2 scores of 0.8650, 0.9177, and 0.9328 on CVC-ClinicDB, 0.7936, 0.8618, and 0.8855 on Kvasir-SEG, and 0.8045, 0.8747 and 0.8909 on BKAI-NeoPolyp datasets respectively. Also, an improved generalization performance was achieved by the ConvSegNet model, compared to the benchmark polyp segmentation models.

HookNet: Efficient Multiscale Context Aggregation for High-Accuracy Detection of Fabric Defects

Automatic defect detection is a critical stage of quality control in the textile industry. Because defects vary in texture structure, size, and spatial distribution, existing defect detection methods have difficulty in achieving a perfect trade-off between detection efficiency, accuracy, and generalizability. This paper presents an advanced fabric defect detection model based on a deep learning algorithm. A novel parallel dilated attention module was designed: this is nested in the deep layers of the neural network backbone to establish global channel dependencies and capture multiscale contextual information. Further, a hook-shaped feature pyramid network was developed for multiscale context aggregation: this directs the network to focus on lower-level features and is helpful for further improving detection efficiency without sacrificing detection accuracy. In addition, the Alpha-GIoU loss is used to improve the accuracy of bounding box regression because of its modulated power parameter α. The advantages and effectiveness of the proposed method were statistically analyzed using data from several public datasets. The code is available at https://github.com/aabb605/HookNet.

ARU-DGAN: A dual generative adversarial network based on attention residual U-Net for magneto-acousto-electrical image denoising.

Magneto-Acousto-Electrical Tomography (MAET) is a multi-physics coupling imaging modality that integrates the high resolution of ultrasound imaging with the high contrast of electrical impedance imaging. However, the quality of images obtained through this imaging technique can be easily compromised by environmental or experimental noise, thereby affecting the overall quality of the imaging results. Existing methods for magneto-acousto-electrical image denoising lack the capability to model local and global features of magneto-acousto-electrical images and are unable to extract the most relevant multi-scale contextual information to model the joint distribution of clean images and noise images. To address this issue, we propose a Dual Generative Adversarial Network based on Attention Residual U-Net (ARU-DGAN) for magneto-acousto-electrical image denoising. Specifically, our model approximates the joint distribution of magneto-acousto-electrical clean and noisy images from two perspectives: noise removal and noise generation. First, it transforms noisy images into clean ones through a denoiser; second, it converts clean images into noisy ones via a generator. Simultaneously, we design an Attention Residual U-Net (ARU) to serve as the backbone of the denoiser and generator in the Dual Generative Adversarial Network (DGAN). The ARU network adopts a residual mechanism and introduces a linear Self-Attention based on Cross-Normalization (CNorm-SA), which is proposed in this paper. This design allows the model to effectively extract the most relevant multi-scale contextual information while maintaining high resolution, thereby better modeling the local and global features of magneto-acousto-electrical images. Finally, extensive experiments on a real-world magneto-acousto-electrical image dataset constructed in this paper demonstrate significant improvements in preserving image details achieved by ARU-DGAN. Furthermore, compared to the state-of-the-art competitive methods, it exhibits a 0.3 dB increase in PSNR and an improvement of 0.47% in SSIM.

Land Use Classification of High-Resolution Multispectral Satellite Images With Fine-Grained Multiscale Networks and Superpixel Postprocessing

Land use recognition from multispectral satellite images is fundamentally critical for geological applications, but the results are not satisfied. The scale dimension of current multiscale learning is too coarse to account for rich scales in multispectral images, and pixel-wise classification tends to produce “salt-and-pepper” labels due to possible misclassification in heterogeneous regions. In this paper, these issues are addressed by proposing a new pixel-wise classification model with finer scales for convolutional neural networks. The model is designed to extract multiscale contextual information using multiscale networks at a fine-grained level, addressing the issue of insufficient multiscale learning for classification. Furthermore, a small-scale segmentation-combination method is introduced as a post-processing solution to smooth fragmented classification results. The proposed method is tested on GF-1, GF-2, DEIMOS-2, GeoEye-1, and Sentinel-2 satellite images, and compared with six neural-network-based algorithms. The results demonstrate the effectiveness of the proposed model in finding objects of large scale difference, improving classification accuracy, and reducing classified fragments. The discussion also illustrates that convolutional neural networks and pixel-wise inference are more practical than Transformer and patch-wise recognition.

Multiple Field-of-View Based Attention Driven Network for Weakly Supervised Common Bile Duct Stone Detection.

Common bile duct (CBD) stones caused diseases are life-threatening. Because CBD stones locate in the distal part of the CBD and have relatively small sizes, detecting CBD stones from CT scans is a challenging issue in the medical domain. We propose a deep learning based weakly-supervised method called multiple field-of-view based attention driven network (MFADNet) to detect CBD stones from CT scans based on image-level labels. Three dominant modules including a multiple field-of-view encoder, an attention driven decoder and a classification network are collaborated in the network. The encoder learns the feature of multi-scale contextual information while the decoder with the classification network is applied to locate the CBD stones based on spatial-channel attentions. To drive the learning of the whole network in a weakly-supervised and end-to-end trainable manner, four losses including the foreground loss, background loss, consistency loss and classification loss are proposed. Compared with state-of-the-art weakly-supervised methods in the experiments, the proposed method can accurately classify and locate CBD stones based on the quantitative and qualitative results. We propose a novel multiple field-of-view based attention driven network for a new medical application of CBD stone detection from CT scans while only image-levels are required to reduce the burdens of labeling and help physicians automatically diagnose CBD stones. The source code is available at https://github.com/nchucvml/MFADNet after acceptance. Our deep learning method can help physicians localize relatively small CBD stones for effectively diagnosing CBD stone caused diseases.

Multiscale Attention Feature Aggregation Network for Cloud and Cloud Shadow Segmentation

Cloud and cloud shadow segmentation is one of the most important problems in remote sensing image processing. Due to the vulnerability to ground object interference, noise interference and other factors, and the lack of generalization ability, traditional deep learning network would inevitably lose details and spatial information, resulting in imprecise segmentation of cloud and cloud shadow boundaries, missing detection and false detection. In order to solve these problems, a multi-scale attention feature aggregation network is proposed, which extracts semantic information at different levels based on residual network. A multi-scale strip pooling attention module is designed to extract multi-scale context information and deep spatial channel information. In order to improve the feature extraction of the model, the deep multi-head feedforward transfer attention module is introduced to enable the adjacent two layers of the backbone network to guide each other for feature mining. Then, a bilateral feature fusion module is used to guide the fusion of low-level semantic information and high-level detail information. Finally, in order to enhance the repair of boundary details of cloud and cloud shadow, a boundary refinement boosting model is designed. The experimental results show that our model can handle complex cloud cover scenes, and has excellent performance on cloud, cloud shadow dataset and CSWV dataset. Its segmentation accuracy is superior to the existing methods, which is of great significance to the related work of cloud detection.

MCA-Unet: A multiscale context aggregation U-Net for the segmentation of COVID-19 lesions from CT images

The pandemic of coronavirus disease (COVID-19) caused the world to face an existential health crisis. COVID-19 lesions segmentation from CT images is nowadays an essential step to assess the severity of the disease and the amount of damage to the lungs. Deep learning has brought about a breakthrough in medical image segmentation where U-Net is the most prominent deep network. However, in this study, we argue that its architecture still lacks in certain aspects. First, there is an incompatibility in the U-Net skip connection between the encoder and decoder features which adversely affects the final prediction. Second, it lacks capturing multiscale context information and ignores the contribution of all semantic information through the segmentation process. Therefore, we propose a model named MCA-Unet, a novel multiscale deep learning segmentation model, which proposes some modifications to improve upon the U-Net model. MCA-Unet is integrated with a multiscale context aggregation module which is constituted of two blocks; a context embedding block (CEB) and a cascaded dilated convolution block (CDCB). The CEB aims at reducing the semantic gap between the concatenated features along the U-Net skip connections, it enriches the low-level encoder features with rich semantics inherited from the subsequent higher-level features, to reduce the semantic gap between the low-processed encoder features and the highly-processed decoder features, thus ensuring effectual concatenation. The CDCB is integrated to address the variability in shape and size of the COVID-19 lesions, it captures global context information by gradually expanding the receptive field, then operates reversely to capture the small fine details that might be scattered by enlarging the receptive field. To validate the robustness of our model, we tested it on a publicly available dataset of 1705 axial CT images with different types of COVID-19 infection. Experimental results show that MCA-Unet has attained a remarkable gain in performance in comparison with the basic U-Net and its variant. It achieved high performance using different evaluation metrics showing 88.6% Dice similarity coefficient, 85.4% Jaccard index, and 93.5% F-score measure. This outperformance shows great potential to help physicians during their examination and improve the clinical workflow.

3D Soma Detection in Large-Scale Whole Brain Images via a Two-Stage Neural Network.

3D soma detection in whole brain images is a critical step for neuron reconstruction. However, existing soma detection methods are not suitable for whole mouse brain images with large amounts of data and complex structure. In this paper, we propose a two-stage deep neural network to achieve fast and accurate soma detection in large-scale and high-resolution whole mouse brain images (more than 1TB). For the first stage, a lightweight Multi-level Cross Classification Network (MCC-Net) is proposed to filter out images without somas and generate coarse candidate images by combining the advantages of the multi convolution layer's feature extraction ability. It can speed up the detection of somas and reduce the computational complexity. For the second stage, to further obtain the accurate locations of somas in the whole mouse brain images, the Scale Fusion Segmentation Network (SFS-Net) is developed to segment soma regions from candidate images. Specifically, the SFS-Net captures multi-scale context information and establishes a complementary relationship between encoder and decoder by combining the encoder-decoder structure and a 3D Scale-Aware Pyramid Fusion (SAPF) module for better segmentation performance. The experimental results on three whole mouse brain images verify that the proposed method can achieve excellent performance and provide the reconstruction of neurons with beneficial information. Additionally, we have established a public dataset named WBMSD, including 798 high-resolution and representative images ( 256 ×256 ×256 voxels) from three whole mouse brain images, dedicated to the research of soma detection, which will be released along with this paper.

C2BA-UNet: A context-coordination multi-atlas boundary-aware UNet-like method for PET/CT images based tumor segmentation.

Tumor segmentation is a necessary step in clinical processing that can help doctors diagnose tumors and plan surgical treatments. Since tumors are usually small, the locations and appearances vary substantially across individuals, and the contrast between tumors and adjacent normal tissues is low, tumor segmentation is still a challenging task. Although convolutional neural networks (CNNs) have achieved good results in tumor segmentation, the information about tumor boundaries has been rarely explored. To solve the problem, this paper proposes a new method for automatic tumor segmentation in PET/CT images based on context-coordination and boundary-aware, termed as C2BA-UNet. We employ a UNet-like backbone network and replace the encoder with EfficientNet-B0 for efficiency. To acquire potential tumor boundaries, we propose a new multi-atlas boundary-aware (MABA) module based on gradient atlas, uncertainty atlas, and level set atlas, that focuses on uncertain regions between tumors and adjacent tissues. Furthermore, we propose a new context coordination module (CCM) to combine multi-scale context information with attention mechanism to optimize skip connection in high-level layers. To validate the superiority of our method, we conduct experiments on a publicly available soft tissue sarcoma (STS) dataset and a lymphoma dataset, and the results show our method is competitive with other comparison methods.

Cross Position Aggregation Network for Few-Shot Strip Steel Surface Defect Segmentation

Strip steel surface defect (S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> D) segmentation is a crucial method to inspect the surface quality of strip steel in the producing-and-manufacturing. However, existing S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> D semantic segmentation methods depend on quite a few labeled defective samples for training, and generalization to novel defect categories that have not yet been trained is challenging. Additionally, some defect categories are incredibly sparse in the industrial production processes. Motivated by the above problems, this paper proposed a simple but effective few-shot segmentation method named cross position aggregation network (CPANet), which intends to learn a network that can segment untrained S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> D categories with only a few labeled defective samples. Using a cross-position proxy (CPP) module, our CPANet can effectively aggregate long-range relationships of discrete defects, and support auxiliary (SA) can further improve the feature aggregation capability of CPP. Moreover, CPANet introduces a space-squeeze attention (SSA) module to aggregate multi-scale context information of defect features and suppresses disadvantageous interference from background information. In addition, a novel S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> D few-shot semantic segmentation dataset FSSD-12 is proposed to evaluate our CPANet. Through extensive comparison experiments and ablation experiments, we explicitly evaluate that our CPANet with the ResNet-50 backbone achieves state-of-the-art performance on dataset FSSD-12. Our dataset and code are available at: https://github.com/VDT-2048/CPANet.

A dense residual U-net for multiple sclerosis lesions segmentation from multi-sequence 3D MR images

Multiple Sclerosis (MS) is an autoimmune disease that causes brain and spinal cord lesions, which magnetic resonance imaging (MRI) can detect and characterize. Recently, deep learning methods have achieved remarkable results in the automated segmentation of MS lesions from MRI data. Hence, this study proposes a novel dense residual U-Net model that combines attention gate (AG), efficient channel attention (ECA), and Atrous Spatial Pyramid Pooling (ASPP) to enhance the performance of the automatic MS lesion segmentation using 3D MRI sequences. First, convolution layers in each block of the U-Net architecture are replaced by residual blocks and connected densely. Then, AGs are exploited to capture salient features passed through the skip connections. The ECA module is appended at the end of each residual block and each downsampling block of U-Net. Later, the bottleneck of U-Net is replaced with the ASSP module to extract multi-scale contextual information. Furthermore, 3D MR images of Fluid Attenuated Inversion Recovery (FLAIR), T1-weighted (T1-w), and T2-weighted (T2-w) are exploited jointly to perform better MS lesion segmentation. The proposed model is validated on the publicly available ISBI2015 and MSSEG2016 challenge datasets. This model produced an ISBI score of 92.75, a mean Dice score of 66.88%, a mean positive predictive value (PPV) of 86.50%, and a mean lesion-wise true positive rate (LTPR) of 60.64% on the ISBI2015 testing set. Also, it achieved a mean Dice score of 67.27%, a mean PPV of 65.19%, and a mean sensitivity of 74.40% on the MSSEG2016 testing set. The results show that the proposed model performs better than the results of some experts and some of the other state-of-the-art methods realized related to this particular subject. Specifically, the best Dice score and the best LTPR are obtained on the ISBI2015 testing set by using the proposed model to segment MS lesions.

Multi-scale and attention enhanced graph convolution network for skeleton-based violence action recognition.

Graph convolution networks (GCNs) have been widely used in the field of skeleton-based human action recognition. However, it is still difficult to improve recognition performance and reduce parameter complexity. In this paper, a novel multi-scale attention spatiotemporal GCN (MSA-STGCN) is proposed for human violence action recognition by learning spatiotemporal features from four different skeleton modality variants. Firstly, the original joint data are preprocessed to obtain joint position, bone vector, joint motion and bone motion datas as inputs of recognition framework. Then, a spatial multi-scale graph convolution network based on the attention mechanism is constructed to obtain the spatial features from joint nodes, while a temporal graph convolution network in the form of hybrid dilation convolution is designed to enlarge the receptive field of the feature map and capture multi-scale context information. Finally, the specific relationship in the different skeleton data is explored by fusing the information of multi-stream related to human joints and bones. To evaluate the performance of the proposed MSA-STGCN, a skeleton violence action dataset: Filtered NTU RGB+D was constructed based on NTU RGB+D120. We conducted experiments on constructed Filtered NTU RGB+D and Kinetics Skeleton 400 datasets to verify the performance of the proposed recognition framework. The proposed method achieves an accuracy of 95.3% on the Filtered NTU RGB+D with the parameters 1.21M, and an accuracy of 36.2% (Top-1) and 58.5% (Top-5) on the Kinetics Skeleton 400, respectively. The experimental results on these two skeleton datasets show that the proposed recognition framework can effectively recognize violence actions without adding parameters.

SegMarsViT: Lightweight Mars Terrain Segmentation Network for Autonomous Driving in Planetary Exploration

Planetary rover systems need to perform terrain segmentation to identify feasible driving areas and surround obstacles, which falls into the research area of semantic segmentation. Recently, deep learning (DL)-based methods were proposed and achieved great performance for semantic segmentation. However, due to the on-board processor platform’s strict comstraints on computational complexity and power consumption, existing DL approaches are almost impossible to be deployed on satellites under the burden of extensive computation and large model size. To fill this gap, this paper targeted studying effective and efficient Martian terrain segmentation solutions that are suitable for on-board satellites. In this article, we propose a lightweight ViT-based terrain segmentation method, namely, SegMarsViT. In the encoder part, the mobile vision transformer (MViT) block in the backbone extracts local–global spatial and captures multiscale contextual information concurrently. In the decoder part, the cross-scale feature fusion modules (CFF) further integrate hierarchical context information and the compact feature aggregation module (CFA) combines multi-level feature representation. Moreover, we evaluate the proposed method on three public datasets: AI4Mars, MSL-Seg, and S5Mars. Extensive experiments demonstrate that the proposed SegMarsViT was able to achieve 68.4%, 78.22%, and 67.28% mIoU on the AI4Mars-MSL, MSL-Seg, and S5Mars, respectively, under the speed of 69.52 FPS.

Neural Network Based on Multi-Scale Saliency Fusion for Traffic Signs Detection

Aiming at recognizing small-scale and complex traffic signs in the driving environment, a traffic sign detection algorithm YOLO-FAM based on YOLOv5 is proposed. Firstly, a new backbone network, ShuffleNet-v2, is used to reduce the algorithm’s parameters, realize lightweight detection, and improve detection speed. Secondly, the Bidirectional Feature Pyramid Network (BiFPN) structure is introduced to capture multi-scale context information, so as to obtain more feature information and improve detection accuracy. Finally, location information is added to the channel attention using the Coordinated Attention (CA) mechanism, thus enhancing the feature expression. The experimental results show that compared with YOLOv5, the mAP value of this method increased by 2.27%. Our approach can be effectively applied to recognizing traffic signs in complex scenes. At road intersections, traffic planners can better plan traffic and avoid traffic jams.

Iterative Convolutional Encoder-Decoder Network with Multi-Scale Context Learning for Liver Segmentation

ABSTRACT Rapid and accurate extraction of liver tissue from abdominal computed tomography (CT) and magnetic resonance (MR) images has critical importance for diagnosis and treatment of hepatic diseases. Due to adjacent organs with similar intensities and anatomical variations between different subjects, the performance of segmentation approaches based on deep learning still has room for improvement. In this study, a novel convolutional encoder-decoder network incorporating multi-scale context information is proposed. The probabilistic map from previous classifier is iteratively fed into the encoder layers, which fuses high-level shape context with low-level appearance features in a multi-scale manner. The dense connectivity is adopted to aggregate feature maps of varying scales from the encoder and decoder. We evaluated the proposed method with 2D and 3D application on abdominal CT and MR images of three public datasets. The proposed method generated liver segmentation with significantly higher accuracy (p <0.05), in comparison to several competing methods. These promising results suggest that the novel model could offer high potential for clinical workflow.

MAGF-Net: A multiscale attention-guided fusion network for retinal vessel segmentation

Retinal fundus images contain plenty of morphological information, so it is particularly important to realize precise segmentation of the retinal vessels for clinical diagnosis. With the rapid development of deep convolutional neural networks (DCNNs), to replace earlier manual labeling methods and reduce the labor cost, DCNN-based automatic segmentation methods have been greatly developed. U-Net and its variant models have obtained superior performance, but segmentation tasks are still challenging for the following reasons: First, features from encoders and decoders are not sufficiently fused to retain more effective information. Second, the limited receptive field will also affect contextual information extraction. In addition, although the continuous pooling operations can speed up the segmentation network training efficiency, they also lose detailed information during the downsampling process. To address the above issues and precisely segment the vessel structures from fundus images, a multiscale attention-guided fusion network, called MAGF-Net, is presented for automatic retinal vessel segmentation. To capture multiscale contextual features, a multiscale attention (MSA) block is proposed to construct the backbone network. Furthermore, a feature enhancement (FE) block is also proposed and embedded in the bottleneck layer to acquire global multiscale contextual information. To take full advantage of the channel information from deep layers and the spatial information from shallow layers, an attention-guided fusion (AGF) block is designed to fuse features from different network layers. Moreover, a hybrid feature pooling (HFP) block is employed to preserve more information during the downsampling operation. To evaluate the segmentation performance of the proposed MAGF-Net, extensive segmentation experiments are conducted on three public datasets: the CHASE_DB1 set, the DRIVE set and the STARE set. The experimental results show that the proposed MAGF-Net can obtain remarkable segmentation performance compared with other advanced methods. In particular, the ability of the proposed MAGF-Net to segment thin blood vessels is significantly improved.