Rough Segmentation Research Articles

Adaptive granulation Renyi rough entropy image thresholding method with nested optimization

The rough entropy based image thresholding algorithm can effectively deal with the uncertainty information in an image. Nevertheless, the existing rough entropy based thresholding methods have two limitations. One is that there is no generalized rough entropy definition. The other is that the granule size cannot be selected automatically in combination with the image information in the granulation process. In order to address these two problems, an adaptive granulation Renyi rough entropy image thresholding method with nested optimization is proposed in this paper. First, a definition of generalized rough entropy with parameter based on Renyi entropy form is proposed to describe the uncertainty information of complex images. Second, in order to select the granule size in combination with the specific image information, a granule size selection method is proposed by maximizing the uniformity of the segmented image regions. Finally, a nested optimization adaptive granulation Renyi rough entropy thresholding segmentation algorithm is proposed. In the experiments, the Renyi rough entropy is compared with the existing four rough entropy in image thresholding. The new parametric rough entropy can obtain better segmentation results than the other existing rough entropy. The novel adaptive granulation Renyi rough entropy thresholding segmentation algorithm is compared with the state-of-the-art image segmentation methods on several different image data sets, which verifies the effectiveness of the novel algorithm.

REU-Net: Region-enhanced nuclei segmentation network

Nuclei segmentation is a key technique for automatic pathological screening. Although many methods have been proposed, it remains a challenge because of numerous nuclei clusters, high variability of object appearances and complex backgrounds. To address these issues, we propose a novel multi-task region-enhanced nuclei segmentation network (REU-Net). It stacks three U-shaped structures by combining serial and parallel approaches to construct a multi-task architecture. The model employs two auxiliary tasks, i.e., contour extraction and rough segmentation to help the main task of fine segmentation. The saliency regions are enhanced by the prediction results of the auxiliary tasks, and the enhanced images are further segmented through the main task. In addition, the spatial and texture features in auxiliary tasks are aggregated by attention gates, helping the main task to refine the details of nuclei and contours. Extensive experiments are conducted to evaluate the proposed method qualitatively and quantitatively. Experimental results show that REU-Net outperforms the state-of-the-art methods on HUSTS, MoNuSeg, CoNSep and CPM-17 datasets.

Cascade Path Augmentation Unet for bladder cancer segmentation in MRI

Treatment choices for patients with bladder cancer (BCa) are determined by the presence of muscular invasion. The precise segmentation of the inner and outer walls (IW and OW), as well as the bladder tumor (BT), is crucial for improving computer-aided diagnosis of muscle-invasive bladder cancer (MIBC). To propose a novel deep learning-based model to improve the segmentation accuracy of the IW, OW, and BT, which can be useful in clinical practice. We proposed a Cascade Path Augmentation Unet (CPA-Unet) network to conduct multi-regional segmentation of the bladder using 1545 T2-weighted MRI scans. The model employs a cascade strategy to eliminate the redundant information in the background. Unet is used to segment the bladder from the background in the rough segmentation. The path augmentation structure is used in the fine segmentation to mine multi-scale features. Additionally, the partial dense connection is adopted as the skip connection module to concatenate the low- and high-level sematic features. The CPA-Unet is trained using 1391 T2WI slices and tested using 154 T2WI slices. In comparison to previous deep learning-based methods, the CPA-Unet achieves superior segmentation results in terms of Dice similarity coefficient (DSC) and Hausdorff distance (HD) (IW: DSC =98.19%, HD =2.07mm; OW: DSC =82.24%, HD =2.62mm; BT: DSC =87.40%, HD =0.76mm). Our proposed CPA-Unet network is capable of segmenting the bladder into its IW and OW, as well as tumors. The segmentation results provide a reliable and effective foundation for computer-assisted clinical diagnosis of MIBC.

CSAUNet: A cascade self-attention u-shaped network for precise fundus vessel segmentation

Vessel segmentation contributes to precise diagnosis of fundus diseases. However, due to the existence of micro vessels and class imbalance, it remains a challenging task to accurately segment vessels from fundus images. To address this issue, a coarse-to-fine framework, namely cascade self-attention U-shaped network (CSAUNet), is proposed in this paper. Firstly, a self-attention U-shaped network (SAUNet) is pre-trained to roughly locate the fundus vessels. Afterwards, the pretrained SAUNet is cascaded with a residual self-attention UNet (Res-SAUNet), and the rough segmentation results of the pretrained SAUNet are regarded as the spatial attention maps to force the Res-SAUNet to focus on significant regions. In addition, self-attention modules (SAMs) are introduced in both the SAUNet and Res-SAUNet to effectively model the long-range dependencies across fundus vessels. The experimental results show that the accuracy, sensitivity, specificity and dice similarity coefficient (DSC) of CSAUNet on three public databases, including DRIVE, STARE and synthetic images (SI), reach 96.76%/83.4%/98.1%/82.07%, 97.28%/83.04%/98.62%/84% and 0.9915/0.9539/0.9962/0.9612, respectively. In addition, the coarse-to-fine framework can improve the DSC and sensitivity on DRIVE, STARE and SI by 1.24%/2.37%, 1.69%/1.04% and 0.8%/0.72%, respectively, and the incorporation of SAMs can further improve the DSC on DRIVE, STARE and SI by 0.37%, 0.7% and 0.49%, respectively, which indicates that CSAUNet can improve the segmentation performance for fundus vessels.

A Robust Image Segmentation Framework Based on Nonlocal Total Variation Spectral Transform

Image segmentation plays an important role in various computer vision tasks. Nevertheless, noise always inevitably appears in images and brings a big challenge to image segmentation. To handle the problem, we study the nonlocal total variation (NLTV) spectral theory and build up an image segmentation framework with NLTV spectral transform to segment images with noise. Firstly, we decompose an image into the NLTV flow in the NLTV spectral transform, with which the max response time of each pixel is calculated. Secondly, a separation surface is constructed with the max response time to distinguish the objects and preserve the structure details in segmentation. Thirdly, the image is filtered by the surface in the NLTV spectral domain, and a rough segmentation result is obtained by means of an inverse transform. Finally, we use a binary process and morphological operations to refine the segmentation result. Experiments illustrate that our method can preserve edge structures effectively and has the ability to achieve competitive segmentation performance compared with the state-of-the-art approaches.

DBFU-Net: Double branch fusion U-Net with hard example weighting train strategy to segment retinal vessel.

BackgroundMany fundus imaging modalities measure ocular changes. Automatic retinal vessel segmentation (RVS) is a significant fundus image-based method for the diagnosis of ophthalmologic diseases. However, precise vessel segmentation is a challenging task when detecting micro-changes in fundus images, e.g., tiny vessels, vessel edges, vessel lesions and optic disc edges.MethodsIn this paper, we will introduce a novel double branch fusion U-Net model that allows one of the branches to be trained by a weighting scheme that emphasizes harder examples to improve the overall segmentation performance. A new mask, we call a hard example mask, is needed for those examples that include a weighting strategy that is different from other methods. The method we propose extracts the hard example mask by morphology, meaning that the hard example mask does not need any rough segmentation model. To alleviate overfitting, we propose a random channel attention mechanism that is better than the drop-out method or the L2-regularization method in RVS.ResultsWe have verified the proposed approach on the DRIVE, STARE and CHASE datasets to quantify the performance metrics. Compared to other existing approaches, using those dataset platforms, the proposed approach has competitive performance metrics. (DRIVE: F1-Score = 0.8289, G-Mean = 0.8995, AUC = 0.9811; STARE: F1-Score = 0.8501, G-Mean = 0.9198, AUC = 0.9892; CHASE: F1-Score = 0.8375, G-Mean = 0.9138, AUC = 0.9879).DiscussionThe segmentation results showed that DBFU-Net with RCA achieves competitive performance in three RVS datasets. Additionally, the proposed morphological-based extraction method for hard examples can reduce the computational cost. Finally, the random channel attention mechanism proposed in this paper has proven to be more effective than other regularization methods in the RVS task.

A robust and high-precision edge segmentation and refinement method for high-resolution images.

Limited by GPU memory, high-resolution image segmentation is a highly challenging task. To improve the accuracy of high-resolution segmentation, High-Resolution Refine Net (HRRNet) is proposed. The network is divided into a rough segmentation module and a refinement module. The former improves DeepLabV3+ by adding the shallow features of 1/2 original image size and the corresponding skip connection to obtain better rough segmentation results, the output of which is used as the input of the latter. In the refinement module, first, the global context information of the input image is obtained by a global process. Second, the high-resolution image is divided into patches, and each patch is processed separately to obtain local details in a local process. In both processes, multiple refine units (RU) are cascaded for refinement processing, and two cascaded residual convolutional units (RCU) are added to the different output paths of RU to improve the mIoU and the convergence speed of the network. Finally, according to the context information of the global process, the refined patches are pieced to obtain the refined segmentation result of the whole high-resolution image. In addition, the regional non-maximum suppression is introduced to improve the Sobel edge detection, and the Pascal VOC 2012 dataset is enhanced, which improves the segmentation accuracy and robust performance of the network. Compared with the state-of-the-art semantic segmentation models, the experimental results show that our model achieves the best performance in high-resolution image segmentation.

SADA-Net: A Shape Feature Optimization and Multiscale Context Information-Based Water Body Extraction Method for High-Resolution Remote Sensing Images

Convolutional neural networks (CNNs) have significance in remote sensing image mapping, and pixel-level representation allows refined results. Due to inconsistencies within a class and different scales of water bodies, the water body mapping has challenges, such as insufficient integrity and rough shape segmentation. To resolve these issues, we proposed an intelligent water bodies extraction method (named SADA-Net) for high-resolution remote sensing images. This method considers multiscale information, context dependence, and shape features. The network framework integrates three critical components: shape feature optimization (SFO), atrous spatial pyramid pooling, and dual attention modules. SADA-Net can accurately extract an extensive range of water bodies in complex scenarios. SADA-Net has certain advantages regarding small and dense water bodies extraction, as the SFO module effectively solves the defects of the unified processing of low-level features in the encoder stage of CNNs, which highlights the shape information of a water body. Two data types (red, green, and blue bands and multispectral images) are employed to verify the performance of the proposed network. The best result achieved an evaluation index F1-Score of 96.14% in large-scale image segmentation, and the structural similarity index measure reached 94.70%. Overall, the proposed method achieves the purpose of maximizing the integrity and optimizing the shape of a water body. Additionally, the SADA-Net proposed in this article has a specific reference value for high-resolution remote sensing image water bodies mapping.

BiAttnNet: Bilateral Attention for Improving Real-Time Semantic Segmentation

Semantic segmentation requires both speed and accuracy. This paper presents a two-branch network BiAttnNet with a unique Bilateral Attention structure that separates all attention modules into the Detail Branch to contribute semantic detail selections for specialized detail exploring. Specifically, the Detail Branch comprises AttnTrans entirely, which provides a better alternate for regular convolution. AttnTrans is a computationally efficient filtration entirely composed of concurrent spatial and channel attention. Meanwhile, a Context Branch is implemented with FCN-ResNet for rough segmentation. By combining two branches’ outputs, BiAttnNet achieves a good balance between speed and accuracy. Evaluations on the Cityscapes testing set conclude that BiAttnNet achieves 74.7% mIoU at 89.2 FPS at a quarter (512 × 1024) resolution with only 2.2 million parameters, running on a single GTX 2080 Ti card.

Verte-Box: A Novel Convolutional Neural Network for Fully Automatic Segmentation of Vertebrae in CT Image.

Due to the complex shape of the vertebrae and the background containing a lot of interference information, it is difficult to accurately segment the vertebrae from the computed tomography (CT) volume by manual segmentation. This paper proposes a convolutional neural network for vertebrae segmentation, named Verte-Box. Firstly, in order to enhance feature representation and suppress interference information, this paper places a robust attention mechanism on the central processing unit, including a channel attention module and a dual attention module. The channel attention module is used to explore and emphasize the interdependence between channel graphs of low-level features. The dual attention module is used to enhance features along the location and channel dimensions. Secondly, we design a multi-scale convolution block to the network, which can make full use of different combinations of receptive field sizes and significantly improve the network’s perception of the shape and size of the vertebrae. In addition, we connect the rough segmentation prediction maps generated by each feature in the feature box to generate the final fine prediction result. Therefore, the deep supervision network can effectively capture vertebrae information. We evaluated our method on the publicly available dataset of the CSI 2014 Vertebral Segmentation Challenge and achieved a mean Dice similarity coefficient of 92.18 ± 0.45%, an intersection over union of 87.29 ± 0.58%, and a 95% Hausdorff distance of 7.7107 ± 0.5958, outperforming other algorithms.

Rethinking CAM in Weakly-Supervised Semantic Segmentation

Weakly supervised semantic segmentation (WSSS) generally utilizes the Class Activation Map (CAM) to synthesize pseudo-labels. However, the current methods of obtaining CAM focus on salient features of a specific layer, resulting in highlighting the most discriminative regions and further leading to rough segmentation results for WSSS. In this paper, we rethink the potential of the ordinary classifier and find that if features of all the layers are applied, the classifier will obtain CAM with complete discriminative regions. Inspired by this, we propose Fully-CAM for WSSS, which can fully exploit the potential of the ordinary classifier and yield more accurate segmentation results. Precisely, Fully-CAM firstly weights feature with their corresponding gradients to yield CAMs of each layer, then fusing these layers’ CAMs could generate an ultimate CAM with complete discriminative regions. Furthermore, Fully-CAM is encapsulated into a plug-in, which can be mounted on any trained ordinary classifier with convolution layer, and it exceeds its previous performance without extra training.

Dual-Branch Network for Cloud and Cloud Shadow Segmentation

Cloud and cloud shadow segmentation is one of the most important issues in remote sensing image processing. Most of the remote sensing images are very complicated. In this work, a dual-branch model composed of Transformer and convolution network is proposed to extract semantic and spatial detail information of the image respectively to solve the problems of false detection and missed detection. To improve the model’s feature extraction, a Mutual Guidance Module is introduced so that the Transformer Branch and the Convolution Branch can guide each other for feature mining. Finally, in view of the problem of rough segmentation boundary, this work uses different features extracted by the Transformer Branch and the Convolution Branch for decoding, and repairs the rough segmentation boundary in the decoding part to make the segmentation boundary clearer. Experimental results on the Landsat-8, Sentinel-2 data, the public dataset HRC_WHU of Wuhan University and the public dataset SPARCS demonstrate the effectiveness of our method and its superiority to existing state-of-the-art Cloud and Cloud Shadow Segmentation approaches.

A cascaded nested network for 3T brain MR image segmentation guided by 7T labeling

Accurate segmentation of the brain into gray matter, white matter, and cerebrospinal fluid using magnetic resonance (MR) imaging is critical for visualization and quantification of brain anatomy. Compared to 3T MR images, 7T MR images exhibit higher tissue contrast that is contributive to accurate tissue delineation for training segmentation models. In this paper, we propose a cascaded nested network (CaNes-Net) for segmentation of 3T brain MR images, trained by tissue labels delineated from the corresponding 7T images. We first train a nested network (Nes-Net) for a rough segmentation. The second Nes-Net uses tissue-specific geodesic distance maps as contextual information to refine the segmentation. This process is iterated to build CaNes-Net with a cascade of Nes-Net modules to gradually refine the segmentation. To alleviate the misalignment between 3T and corresponding 7T MR images, we incorporate a correlation coefficient map to allow well-aligned voxels to play a more important role in supervising the training process. We compared CaNes-Net with SPM and FSL tools, as well as four deep learning models on 18 adult subjects and the ADNI dataset. Our results indicate that CaNes-Net reduces segmentation errors caused by the misalignment and improves segmentation accuracy substantially over the competing methods.

Interpretable Fine-grained BI-RADS Classification of Breast Tumors.

Fine-grained classification of breast tumors is crucial for early diagnosis and timely treatment. Most fine-grained visual classification approaches focus on learning 'informative' visual patterns, which depend on the attention of the network, instead of 'discriminative' patterns, which interpretably contribute to classification. In this paper, we propose to extract discriminative patterns from informative patterns by utilizing the prior information of the dataset. The proposed method can detect the rough contour of the tumor area without boundary ground-truth guidance. At the same time, different masks are generated from the rough contour to reflect prior information on breast cancer. Moreover, a soft-labeling approach is utilized to replace the original BI-RADS label. Our model is trained using image-level object labels and interprets its results via a rough segmentation of tumor parts. Extensive experiments show that our approach achieves a significant performance increase on our BI-RADS classification dataset.

Application of image processing technology in cable detection of underwater robot

Manual inspection of underwater cables has many problems, such as heavy labor intensity, high operation difficulty and low work efficiency. It is more effective to use underwater camera to capture images of underwater cables than manual inspection. The addition of subsea detection and construction equipment raises bottom sediment in the water and causes the water to become turbid and cloudy, it also reduces the contrast of the underwater image and adds irrelevant background information. Therefore, a method of cable location in turbid waters based on image processing is proposed. The method firstly applies the dark channel prior dehazing algorithm. An adaptive histogram equalization algorithm is used to improve the light distribution of underwater image after defogging. Then, we use the local adaptive dynamic threshold segmentation method to achieve the rough segmentation, and the fine segmentation of cable image is carried out by selecting each connected domain region. Finally, the cable pixel coordinates are obtained by the method of center line extraction and frequency domain transformation. Experimental result shows that the proposed algorithm has good effects in terms of clarity.

Current and Future Challenges of Functional MRI and Diffusion Tractography in the Surgical Setting: From Eloquent Brain Mapping to Neural Plasticity

Decades ago, Spetzler (1986) and Sawaya (1998) provided a rough brain segmentation of the eloquent areas of the brain, aimed to help surgical decisions in cases of vascular malformations and tumors, respectively. Currently in clinical use, their criteria are in need of revision. Defining functions (eg, sensorimotor, language and visual) that should be preserved during surgery seems a straightforward task. In practice, locating the specific areas that could cause a permanent vs transient deficit is not an easy task. This is particularly true for the associative cortex and cognitive domains such as language. The old model, with Broca's and Wernicke's areas at the forefront, has been superseded by a dual-stream model of parallel language processing; named ventral and dorsal pathways. This complicated network of cortical hubs and subcortical white matter pathways needing preservation during surgery is a work in progress. Preserving not only cortical regions but most importantly preserving the connections, or white matter fiber bundles, of core regions in the brain is the new paradigm. For instance, the arcuate fascicululs and inferior fronto-occipital fasciculus are key components of the dorsal and ventral language pathways, respectively; and their damage result in permanent language deficits. Interestedly, the damage of the temporal portions of these bundles -where there is a crossroad with other multiple bundles-, appears to be more important (permanent) than the damage of the frontal portions - where plasticity and contralateral activation could help. Although intraoperative direct cortical and subcortical stimulation have contributed largely, advanced MR techniques such as functional MRI (fMRI) and diffusion tractography (DT), are at the epi-center of our current understanding. Nevertheless, these techniques posse important challenges: such as neurovascular uncoupling or venous bias on fMRI; and appropriate anatomical validation or accurate representation of crossing fibers on DT. These limitations should be well understood and taken into account in clinical practice. Unifying multidisciplinary research and clinical efforts is desirable, so these techniques could contribute more efficiently not only to locate eloquent areas but to improve outcomes and our understanding of neural plasticity. Finally, although there are constant anatomical and functional regions at the individual level, there is a known variability at the inter-individual level. This concept should strengthen the importance of a personalized approach when evaluating these regions on fMRI and DT. It should strengthen the importance of personalized treatments as well, aimed to meet tailored needs and expectations.

Aggregation-and-Attention Network for brain tumor segmentation

BackgroundGlioma is a malignant brain tumor; its location is complex and is difficult to remove surgically. To diagnosis the brain tumor, doctors can precisely diagnose and localize the disease using medical images. However, the computer-assisted diagnosis for the brain tumor diagnosis is still the problem because the rough segmentation of the brain tumor makes the internal grade of the tumor incorrect.MethodsIn this paper, we proposed an Aggregation-and-Attention Network for brain tumor segmentation. The proposed network takes the U-Net as the backbone, aggregates multi-scale semantic information, and focuses on crucial information to perform brain tumor segmentation. To this end, we proposed an enhanced down-sampling module and Up-Sampling Layer to compensate for the information loss. The multi-scale connection module is to construct the multi-receptive semantic fusion between encoder and decoder. Furthermore, we designed a dual-attention fusion module that can extract and enhance the spatial relationship of magnetic resonance imaging and applied the strategy of deep supervision in different parts of the proposed network.ResultsExperimental results show that the performance of the proposed framework is the best on the BraTS2020 dataset, compared with the-state-of-art networks. The performance of the proposed framework surpasses all the comparison networks, and its average accuracies of the four indexes are 0.860, 0.885, 0.932, and 1.2325, respectively.ConclusionsThe framework and modules of the proposed framework are scientific and practical, which can extract and aggregate useful semantic information and enhance the ability of glioma segmentation.

Efficient Multi-Organ Multi-Center Cell Nuclei Segmentation Method Based on Deep Learnable Aggregation Network

Automated cell nuclei delineation in whole-slide imaging (WSI) is a fundamental step for many tasks like cancer cell recognition, cancer grading, and cancer subtype classification. Although numerous computational methods have been proposed for segmenting nuclei in WSI images based on image processing and deep learning, existing approaches face major challenges such as color variation due to the use of different stains, the various structures of cell nuclei, and the overlapping and clumped cell nuclei. To circumvent these challenges in this article, we propose an efficient and accurate cell nuclei segmentation method based on deep learning, in which a set of accurate individual cell nuclei segmentation models are developed to predict rough segmentation masks, and then a learnable aggregation network (LANet) is used to predict the final nuclei masks. Besides, we develop cell nuclei segmentation software (with a graphical user interface—GUI) that includes the proposed method and other deep-learning-based cell nuclei segmentation methods. A challenging WSI dataset collected from different centers and organs is used to demonstrate the efficiency of our method. The experimental results reveal that our method obtains a competitive performance compared to the existing approaches in terms of the aggregated Jaccard index (AJI=89.25%) and F1-score (F1=73.02%). The developed nuclei segmentation software can be downloaded from https://github.com/loaysh2010/Cell-Nuclei-Segmentation-GUI-Application.

A CONCEPT FOR THE SEGMENTATION OF INDIVIDUAL URBAN TREES FROM DENSE MLS POINT CLOUDS

Abstract. In our daily lives, trees can be seen as the tallest and most noticeable representatives of the plant kingdom. Especially in urban areas, the individual tree is of high significance and responsible for a manifold of positive effects on the environment and residents. In the context of urban tree registers and thus monitoring of urban vegetation, we propose a general concept for the segmentation of trees from 3D point clouds. Mobile Laser Scanning (MLS) is introduced as the preferred sensor. Based on an analysis of earlier work in this field, we gather arguments and methods in order to involve segmentation in the bigger frame of a tree register workflow, including detailed modeling and change detection. Our concept for segmentation is based on a voxel-structure. In a first step, region growing approaches are used for ground removal and rough segmentation. Later, graph-based optimization will separate neighboring trees. For now, only the general concept can be introduced—quantitative analysis and optimization of the steps will follow in future work.

Edge Detection of Agricultural Products Based on Morphologically Improved Canny Algorithm

The traditional canny edge detection algorithm has its limitations in the aspect of antinoise interference, and it is susceptible to factors such as light. To solve these defects, the Canny algorithm based on morphological improvement was proposed and applied to the detection of agricultural products. First, the algorithm uses the open and close operation of morphology to form a morphological filter instead of the Gaussian filter, which can remove the image noise and strengthen the protection of image edge. Second, the traditional Canny operator is improved to increase the horizontal and vertical templates to 45° and 135° to improve the edge positioning of the image. Finally, the adaptive threshold segmentation method is used for rough segmentation, and on this basis, double detection thresholds are used for further segmentation to obtain the final edge points. The experimental results show that compared with the traditional algorithm applied to the edge detection of agricultural products, this algorithm can effectively avoid the false contour caused by illumination and other factors and effectively improve the antinoise interference while more accurate and fine detection of the edge of real agricultural products.