A Context-Oriented Multi-Scale Neural Network for Fire Segmentation

Pathologists need to comprehensively observe histopathological images of different scales in clinical melanoma diagnosis because histopathological images have various distinctive features at different scales. In clinical diagnosis, the combination of 40X and 10X pathological images is the most commonly used by pathologists. ResNet50 is a popular convolutional neural network structure, which is often used in automatic diagnosis. However, the current research on deep learning-based histopathological image recognition mainly focuses on single-scale networks, and there are few studies on multi-scale neural networks. Therefore, we propose a novel multi-scale convolutional network based on ResNet50. It can merge features of different scales by simultaneously input images of 40x and 10x. Furthermore, we collected 2,241 Whole slide images and constructed a multi-center melanoma dataset for model training and testing. In the experiment, the accuracy of the multi-scale network is 96.4%, and the accuracy of single-scale networks is 90.3%∼95.4%. The multi-scale network has achieved significantly better performance than single-scale networks. Experimental results show that, compared with single-scale networks, multi-scale neural networks can learn more sufficient features and obtain higher diagnostic accuracy.

Cardiomyopathy is one of the most serious public health threats. The precise structural and functional cardiac measurement is an essential step for clinical diagnosis and follow-up treatment planning. Cardiologists are often required to draw endocardial and epicardial contours of the left ventricle (LV) manually in routine clinical diagnosis or treatment planning period. This task is time-consuming and error-prone. Therefore, it is necessary to develop a fully automated end-to-end semantic segmentation method on cardiac magnetic resonance (CMR) imaging datasets. However, due to the low image quality and the deformation caused by heartbeat, there is no effective tool for fully automated end-to-end cardiac segmentation task. In this work, we propose a multi-scale segmentation network (MSSN) for left ventricle segmentation. It can effectively learn myocardium and blood pool structure representations from 2D short-axis CMR image slices in a multi-scale way. Specifically, our method employs both parallel and serial of dilated convolution layers with different dilation rates to capture multi-scale semantic features. Moreover, we design graduated up-sampling layers with subpixel layers as the decoder to reconstruct lost spatial information and produce accurate segmentation masks. We validated our method using 164 T1 Mapping CMR images and showed that it outperforms the advanced convolutional neural network (CNN) models. In validation metrics, we archived the Dice Similarity Coefficient (DSC) metric of 78.96%.

Neural networks are now widely and successfully used in the recognition of handwritten numerals. Despite their wide use in recognition, neural networks have not seen widespread use in segmentation. Segmentation can be extremely difficult in the presence of connected numerals, fragmented numerals, and background noise, and its failure is a principal cause of rejected and incorrectly read documents. Therefore, strategies leading to the successful application of neural technologies to segmentation are likely to yield important performance benefits. In this paper we identify problems that have impeded the use of neural networks in segmentation and describe an evolutionary approach to applying neural networks in segmentation. Our approach, based upon the use of monotonic fuzzy valued decision functions computed by feed-forward neural networks, has been successfully employed in a production system.

In recent years, image segmentation techniques based on deep learning have achieved many applications in remote sensing, medical, and autonomous driving fields. In space exploration, the segmentation of spacecraft objects by monocular images can support space station on-orbit assembly tasks and space target position and attitude estimation tasks, which has essential research value and broad application prospects. However, there is no segmentation network designed for spacecraft targets. This paper proposes an end-to-end spacecraft image segmentation network using the semantic segmentation network DeepLabv3+ as the basic framework. We develop a multi-scale neural network based on sparse convolution. First, the feature extraction capability is improved by the dilated convolutional network. Second, we introduce the channel attention mechanism into the network to recalibrate the feature responses. Finally, we design a parallel atrous spatial pyramid pooling (ASPP) structure that enhances the contextual information of the network. To verify the effectiveness of the method, we built a spacecraft segmentation dataset on which we conduct experiments on the segmentation algorithm. The experimental results show that the encoder+ attention+ decoder structure proposed in this paper, which focuses on high-level and low-level features, can obtain clear and complete masks of spacecraft targets with high segmentation accuracy. Compared with DeepLabv3+, our method is a significant improvement. We also conduct an ablation study to research the effectiveness of our network framework.

Breast cancer is currently the second most fatal cancer in women, but timely diagnosis and treatment can reduce its mortality. Breast masses are the most obvious means of cancer identification, and thus, accurate segmentation of masses is critical. In contrast to mass-centered patch segmentation, accurate segmentation of breast masses in full-field mammograms is always a challenging topic because of the extremely low signal-to-noise ratio and the uncertainty with respect to the shape, size, and location of the mass. In this study, we propose a novel adaptive channel and multiscale spatial context network for breast mass segmentation in full-field mammograms. A standard encoder-decoder structure is employed, and an elaborate adaptive channel and multiscale spatial context module (ACMSC module) is embedded in a multilevel manner in our network for accurate mass segmentation. The proposed ACMSC module utilizes the self-attention mechanism to adaptively capture discriminative contextual information among channel and spatial dimensions.The multilevel embedding of the ACMSC module enables the network to learn distinguishing features on multiple scales of feature maps. Our proposed model is evaluated on two public datasets, CBIS-DDSM and INbreast. The experimental results show that by adaptively capturing the context of the channel and spatial dimensions, our model can effectively remove false positives, predict difficult samples and achieve state-of-the-art results, with Dice coefficients of 82.81% for CBIS-DDSM and 84.11% for INbreast, respectively. We hope that our work will contribute to the CAD system for breast cancer diagnosis and ultimately improve clinical diagnosis.

Semantic segmentation methods based on deep learning have provided the state-of-the-art performance in recent years. Based on deep learning, many Convolutional Neural Network (CNN) models have been proposed. Among them, U-Net with the simple encoder and decoder structure, can learn multi-scale features with various context information and has become one of the most popular neural network architectures for medical image segmentation. To reuse the features with the detail image structure in the encoder path, U-Net utilizes a skip-connection structure to simply copy the low-level features in the encoder to the decoder, and cannot explore the correlations between two paths and different scales. This study proposes a multi-scale context interaction learning network (MCIU-net) for medical image segmentation. First, to effectively fuse the features with detail structure in the encoder path and more semantic information in the decoder path, we conduct interaction learning on the corresponding scale via the bi-directional ConvLSTM (BConvLSTM) unit. Second, the interaction learning among all blocks of the decoder path is also employed for dynamically merging multi-scale contexts. We validate our proposed interaction learning network on three medical image datasets: retinal blood vessel segmentation, skin lesion segmentation, and lung segmentation, and demonstrate promising results compared with the state-of-the-art methods.

NURBS curve shape prior-guided multiscale attention network for automatic segmentation of the inferior alveolar nerve.

Weed control has always been one of the most important issues in agriculture. The research based on deep learning methods for weed identification and segmentation in the field provides necessary conditions for intelligent point-to-point spraying and intelligent weeding. However, due to limited and difficult-to-obtain agricultural weed datasets, complex changes in field lighting intensity, mutual occlusion between crops and weeds, and uneven size and quantity of crops and weeds, the existing weed segmentation methods are unable to perform effectively. In order to address these issues in weed segmentation, this study proposes a multi-scale convolutional attention network for crop and weed segmentation. In this work, we designed a multi-scale feature convolutional attention network for segmenting crops and weeds in the field called MSFCA-Net using various sizes of strip convolutions. A hybrid loss designed based on the Dice loss and focal loss is used to enhance the model’s sensitivity towards different classes and improve the model’s ability to learn from hard samples, thereby enhancing the segmentation performance of crops and weeds. The proposed method is trained and tested on soybean, sugar beet, carrot, and rice weed datasets. Comparisons with popular semantic segmentation methods show that the proposed MSFCA-Net has higher mean intersection over union (MIoU) on these datasets, with values of 92.64%, 89.58%, 79.34%, and 78.12%, respectively. The results show that under the same experimental conditions and parameter configurations, the proposed method outperforms other methods and has strong robustness and generalization ability.

The information in remote sensing images often leads to incomplete building contours and suboptimal adaptability to complex building scenes. To address these issues, we propose a novel multi-scale network with dual attention mechanisms to extract clear building boundaries. The Squeeze-and-Excitation (SE) module is employed to bolster feature extraction, and the Atrous Spatial Pyramid Pooling (ASPP) module is integrated to capture multi-scale feature information. Then, in the decoding phase, channel grouping shuffle and dual attention mechanisms are synergistically integrated to exploit the interrelations and global dependencies of building features. Finally, a hybrid loss function is devised to address the class imbalance and thereby ensure more stable network training. Experimental evaluations on two high-resolution remote sensing datasets, Zimbabwe and Massachusetts, demonstrate that the proposed method markedly surpasses the performance of semantic segmentation networks such as PSPnet, U-net, and DAnet in terms of accuracy, recall, F1 score, and Mean Intersection over Union (MIoU), achieving an F1 score of up to 83.23% and an MIoU of 73.56%. This multi-scale attention network holds substantial promise for practical applications in building extraction.

Robust and efficient abdominal CT segmentation using shape constrained multi-scale attention network

Breast tumor segmentation is a critical aspect of magnetic resonance imaging (MRI)-based breast disease diagnosis. Numerous networks and algorithms, including U-Net and its enhancements, have been proposed for breast tumor segmentation. However, existing methods have certain shortcomings and limitations, including insufficient extraction of multi-scale contextual information, which poses challenges in adapting to tumors of different sizes and distinguishing tumor boundaries from surrounding tissues. Additionally, the feature extraction process lacks specificity and is prone to interference from irrelevant information outside the tumor region. This study aimed to address these challenges, and achieve the accurate and automated segmentation of breast tumors in MRI scans. A new three-dimensional (3D) breast tumor segmentation network named the multi-scale hybrid attention U-shaped network (MHAU-Net) was designed. The network used four sets of atrous convolutions with different dilated ratios to extract multi-scale context information. Global pooling and single-channel convolution structures were employed to construct channel and spatial blocks. Subsequently, the network integrated four sets of atrous convolutions with spatial and channel attention blocks to extract hybrid attention features. Compared to existing MRI segmentation networks for breast tumors, the MHAU-Net demonstrated superior performance in extracting informative features and adapting to tumors of diverse sizes and shapes. To evaluate the proposed approach, we curated a large-scale breast MRI dataset comprising 906 3D images. A comparative analysis with seven commonly used segmentation networks revealed the superior performance of our method. Our network had a dice similarity coefficient (DSC) and intersection over union (IoU) of 84.1%±2.1% and 74.2%±3.4%, respectively, representing a 6.0% and 7.1% improvement over the baseline 3D U-Net. Additionally, our method had DSC values of 85.7%±1.6%, 84.3%±2.8%, 86.7%±1.7%, and 86.3%±1.5% for single, small, large, and mass tumors, respectively. Our results highlight the superior overall performance of the proposed method, and show its ability to adapt to various types of tumor images. This study establishes a solid foundation for further exploring the application value of deep learning in breast cancer diagnosis.

Microbial keratitis is an infection of the cornea of the eye that is commonly caused by prolonged contact lens wear, corneal trauma, pre-existing systemic disorders and other ocular surface disorders. It can result in severe visual impairment if improperly managed. According to the latest World Vision Report, at least 4.2 million people worldwide suffer from corneal opacities caused by infectious agents such as fungi, bacteria, protozoa and viruses. In patients with fungal keratitis (FK), often overt symptoms are not evident, until an advanced stage. Furthermore, it has been reported that clear discrimination between bacterial keratitis and FK is a challenging process even for trained corneal experts and is often misdiagnosed in more than 30% of the cases. However, if diagnosed early, vision impairment can be prevented through early cost-effective interventions. In this work, we propose a multi-scale convolutional neural network (MS-CNN) for accurate segmentation of the corneal region to enable early FK diagnosis. The proposed approach consists of a deep neural pipeline for corneal region segmentation followed by a ResNeXt model to differentiate between FK and non-FK classes. The model trained on the segmented images in the region of interest, achieved a diagnostic accuracy of 88.96%. The features learnt by the model emphasize that it can correctly identify dominant corneal lesions for detecting FK.

Lung area segmentation is an essential step for disease analysis on thoracic computed tomography (CT) scans, which provide helpful information in visual inspection by physicians as well as in quantitative analysis by computer. In this work, we investigated deep convolutional neural network (DCNN) for lung area segmentation. With U-Net [1] and V-Net [2] (which can be regarded as a 3D version of U-Net) as the baseline network, a new multi-scale prediction network (MPN) was designed and evaluated with dice coefficient, Jaccard index, which is also known as Intersect over Union (IoU), Hausdorff distance and coverage rate of nodule areas as criteria. It was found that MPN achieved dice coefficient, Jaccard index, Hausdorff distance, and coverage rate of 0.9845, 0.9697, 3.7480, and 87.54% while the corresponding values for baseline network were 0.9830, 0.9669, 3.9381, and 85.18% (U-Net) and 0.9783, 0.9582, 4.4461 and 52.26% (V-Net).

MRAU-net: Multi-scale residual attention U-shaped network for medical image segmentation

Accurate segmentation of brain glioma is a critical prerequisite for clinical diagnosis, surgical planning and treatment evaluation. In current clinical workflow, physicians typically perform delineation of brain tumor subregions slice-by-slice, which is more susceptible to variabilities in raters and also time-consuming. Besides, even though convolutional neural networks (CNNs) are driving progress, the performance of standard models still have some room for furtherimprovement. To deal with these issues, this paper proposes an attention-guided multi-scale context aggregation network (AMCA-Net) for the accurate segmentation of brain glioma in the magnetic resonance imaging (MRI) images withmulti-modalities. AMCA-Net extracts the multi-scale features from the MRI images and fuses the extracted discriminative features via a self-attention mechanism for brain glioma segmentation. The extraction is performed via a series of down-sampling, convolution layers, and the global context information guidance (GCIG) modules are developed to fuse the features extracted for contextual features. At the end of the down-sampling, a multi-scale fusion (MSF) module is designed to exploit and combine all the extracted multi-scale features. Each of the GCIG and MSF modules contain a channel attention (CA) module that can adaptively calibrate feature responses and emphasize the most relevant features. Finally, multiple predictions with different resolutions are fused through different weightings given by a multi-resolution adaptation (MRA) module instead of the use of averaging or max-pooling to improve the final segmentationresults. Datasets used in this paper are publicly accessible, that is, the Multimodal Brain Tumor Segmentation Challenges 2018 (BraTS2018) and 2019 (BraTS2019). BraTS2018 contains 285 patient cases and BraTS2019 contains 335 cases. Simulations show that the AMCA-Net has better or comparable performance against that of the other state-of-the-art models. In terms of the Dice score and Hausdorff 95 for the BraTS2018 dataset, 90.4% and 10.2 mm for the whole tumor region (WT), 83.9% and 7.4 mm for the tumor core region (TC), 80.2% and 4.3 mm for the enhancing tumor region (ET), whereas the Dice score and Hausdorff 95 for the BraTS2019 dataset, 91.0% and 10.7 mm for the WT, 84.2% and 8.4 mm for the TC, 80.1% and 4.8 mm for the ET. The proposed AMCA-Net performs comparably well in comparison to several state-of-the-art neural net models in identifying the areas involving the peritumoral edema, enhancing tumor, and necrotic and non-enhancing tumor core of brain glioma, which has great potential for clinical practice. In future research, we will further explore the feasibility of applying AMCA-Net to other similar segmentationtasks.