Initial Segmentation Map Research Articles

Polyper: Boundary Sensitive Polyp Segmentation

We present a new boundary sensitive framework for polyp segmentation, termed Polyper.Our method is motivated by a clinical approach that seasoned medical practitioners often leverage the inherent features of interior polyp regions to tackle blurred boundaries.Inspired by this, we propose to explicitly leverages boundary regions to bolster the model's boundary discrimination capability while minimizing computational resource wastage. Our approach first extracts low-confidence boundary regions and high-confidence prediction regions from an initial segmentation map through differentiable morphological operators.Then, we design the boundary sensitive attention that concentrates on augmenting the features near the boundary regions using the high-confidence prediction region's characteristics to generate good segmentation results.Our proposed method can be seamlessly integrated with classical encoder networks, like ResNet-50, MiT-B1, and Swin Transformer.To evaludate the effectiveness of Polyper, we conduct experiments on five publicly available challenging datasets, and receive state-of-the-art performance on all of them. Code is available at https://github.com/haoshao-nku/medical_seg.git.

DRNet: Dual-stage refinement network with boundary inference for RGB-D semantic segmentation of indoor scenes

Semantic segmentation is a dense pixel prediction task, and its accuracy depends on the extraction of long-range contextual knowledge and refinement of segmentation boundaries. Most segmentation methods are based on feature extraction using a convolutional neural network, and layer-by-layer sampling and fusion are applied to solve inherent problems such as chaotic boundaries and scattered objects. Owing to the limited receptive field and loss of details during downsampling, the segmentation results may be unsatisfactory. To address existing shortcomings, we propose a dual-stage refinement network (DRNet) for semantic segmentation. In the first stage, we adopt an efficient spatiotemporal representation learning framework called UniFormer. We also use a novel boundary extractor and initial segmentation map generator to obtain rough segmentation results. In the second stage, we use the rough segmentation map and extracted boundary information in a graph reasoning module that restores the class boundary features while completing global modeling and local information inference. Benefiting from the acquisition of long-range dependencies between image pixels, contextual information promotes the distinction of pixel categories. In addition, edge information can increase the interclass distinguishability and refine the segmentation boundaries. Results from extensive experiments demonstrate that the proposed DRNet outperforms state-of-the-art semantic segmentation methods. The codes and results are available at: https://github.com/EnquanYang2022/DRNet.

TD-Net: A Hybrid End-to-End Network for Automatic Liver Tumor Segmentation From CT Images.

Liver tumor segmentation plays an essential role in diagnosis and treatment of hepatocellular carcinoma or metastasis. However, accurate and automatic tumor segmentation remains a challenging task, owing to vague boundaries and large variations in shapes, sizes, and locations of liver tumors. In this paper, we propose a novel hybrid end-to-end network, called TD-Net, which incorporates Transformer and direction information into convolution network to segment liver tumor from CT images automatically. The proposed TD-Net is composed of a shared encoder, two decoding branches, four skip connections, and a direction guidance block. The shared encoder is utilized to extract multi-level feature information, and the two decoding branches are respectively designed to produce initial segmentation map and direction information. To preserve spatial information, four skip connections are used to concatenate each encoder layer and its corresponding decoder layer, and in the fourth skip connection a Transformer module is constructed to extract global context. Furthermore, a direction guidance block is well-designed to rectify feature maps to further improve segmentation accuracy. Extensive experiments conducted on public LiTS and 3DIRCADb datasets validate that the proposed TD-Net can effectively segment liver tumor from CT images in an end-to-end manner and its segmentation accuracy surpasses those of many existing methods.

DBMF: Dual Branch Multiscale Feature Fusion Network for polyp segmentation

Accurate and reliable segmentation of colorectal polyps is important for the diagnosis and treatment of colorectal cancer. Most of the existing polyp segmentation methods innovatively combine CNN with Transformer. Due to the single combination approach, there are limitations in establishing connections between local feature information and utilizing global contextual information captured by Transformer. Still not a better solution to the problems in polyp segmentation. In this paper, we propose a Dual Branch Multiscale Feature Fusion Network for Polyp Segmentation, abbreviated as DBMF, for polyp segmentation to achieve accurate segmentation of polyps. DBMF uses CNN and Transformer in parallel to extract multi-scale local information and global contextual information respectively, with different regions and levels of information to make the network more accurate in identifying polyps and their surrounding tissues. Feature Super Decoder (FSD) fuses multi-level local features and global contextual information in dual branches to fully exploit the potential of combining CNN and Transformer to improve the network’s ability to parse complex scenes and the detection rate of tiny polyps. The FSD generates an initial segmentation map to guide the second parallel decoder (SPD) to refine the segmentation boundary layer by layer. SPD consists of a multi-scale feature aggregation module (MFA) and parallel polarized self-attention (PSA) and reverse attention fusion modules (RAF). MFA aggregates multi-level local feature information extracted by CNN Brach to find consensus regions between multiple scales and improve the network’s ability to identify polyp regions. PSA uses dual attention to enhance the fine-grained nature of segmented regions and reduce the redundancy introduced by MFA and interference information. RAF mines boundary cues and establishes relationships between regions and boundary cues. The three RAFs guide the network to explore lost targets and boundaries in a bottom-up manner. We used the CVC-ClinicDB, Kvasir, CVC-300, CVC-ColonDB, and ETIS datasets to conduct comparison experiments and ablation experiments between DBMF and mainstream polyp segmentation networks. The results showed that DBMF outperformed the current mainstream networks on five benchmark datasets.

Automated polyp segmentation in colonoscopy images via deep network with lesion-aware feature selection and refinement

Clinically, proper polyp localization in colonoscopy images is crucial for early diagnosis and follow-up treatment of colorectal cancer. However, visual inspection is subjective, error-prone, and burdensome. In this paper, we propose an automated polyp segmentation method (named LFSRNet) to assist physicians to accurately segment polyps in colonoscopy images. The proposed LFSRNet follows an encoder–decoder architecture and benefits from two pivotal modules, i.e., a lesion-aware feature selection module (LFSM) and a lesion-aware feature refinement module (LFRM). Specifically, the LFSM selects lesion-aware features from the top-three highest layers of the encoder via a non-local attention mechanism and fuses them to generate the initial segmentation map for the decoder. The LFRM embedded in the decoder incorporates the guided context information and the output of LFRM from the adjacent higher layer to refine the lesion-aware features. Through top-down deep supervision, our LFSRNet can adaptively select and refine lesion-aware features and precisely localize the polyp regions. Experimental results on the Kvasir-SEG dataset (with the 80%–20% train-test split) show that LFSRNet is superior to six state-of-the-art competing methods and achieves a dice score of 0.9127, an intersection-over-union score of 0.8615, a sensitivity score of 0.9174, an accuracy score of 0.9728, an F2 score of 0.9123, and an MAE score of 0.0291, respectively. More extensive results show that LFSRNet also holds better generalization than competing methods when trained on the Kvasir-SEG dataset and tested on both the CVC-ClinicDB dataset and EndoScene dataset.

Attention based multi-scale parallel network for polyp segmentation

Colonoscopy is an effective method for detecting colorectal polyps and preventing colorectal cancer. Therefore, in clinical practice, it is very important to accurately segment the location and shape of polyps from colorectal images, which can effectively assist clinicians in their diagnosis. However, the varying sizes and shapes of colorectal polyps and the fact that the polyps to be segmented are very small and closely resemble their surroundings make this a challenging task. To address these challenges, we propose a parallel network for multi-scale attention decoding–AMNet. We first perform multi-scale fusion of the high-level feature information extracted from the backbone network using upsampling and downsampling, while aggregating the high-level feature information to generate an initial predictive segmentation map for subsequent contextual guidance. Using a parallel attention module as well as a reverse fusion module, relationships between regions and boundaries are established to further refine the edge information and improve the accuracy of the segmentation. Through extensive experiments on four publicly available polyp segmentation datasets, it has been demonstrated that our AMNet is effective in improving the accuracy of polyp segmentation.

Multi-focus image fusion using learning based matting with sum of the Gaussian-based modified Laplacian

In most traditional multi-focus image fusion algorithms, the focus measures used to detect the focused areas of multiple images are unstable and sensitive to noise. Although many modified methods implement more sophisticated strategies to cope with this problem, the complexity of these methods turns out to be a problem for mobile devices. In order to cope with these problems, a novel multi-focus image fusion method based on image matting technique is put forward in this paper. This method is simple yet effective. Firstly, a new focus measure named the sum of the Gaussian-based modified Laplacian (SGML) is proposed to estimate the focus map of source images. Then, the initial segmentation map can be obtained via a novel sliding window strategy. Further, with the rough segmentation map as an input, learning based image matting technique is performed to extract the exact boundaries between focused and defocused regions. Finally, by combining the information of the focus areas, an all-in-focus image can be obtained. The numerous experiments have revealed that the proposed approach yielded better results in comparison with some competing techniques both in subjective and objective evaluation.

Vessel segmentation and catheter detection in X-ray angiograms using superpixels.

Coronary artery disease (CAD) is the leading cause of death around the world. One of the most common imaging methods for diagnosing CAD is the X-ray angiography (XRA). Diagnosing using XRA images is usually challenging due to some reasons such as, non-uniform illumination, low contrast, presence of other body tissues, and presence of catheter. These challenges make the diagnosis task hard and more prone to misdiagnosis. In this paper, we propose a new method for coronary artery segmentation, catheter detection, and centerline extraction in X-ray angiography images. For the segmentation, initially, three different superpixel scales are exploited, and a measure for vesselness probability of each superpixel is determined. A voting mechanism is used for obtaining an initial segmentation map from the three superpixel scales. The initial segmentation is refined by finding the orthogonal line on each ridge pixel of vessel region. The catheter is detected in the first frame of the angiography sequence and is tracked in other frames by fitting a second order polynomial on it. Also, we use the image ridges for extracting the coronary artery centerlines. We evaluated and compared our method with one of the previous well-known coronary artery segmentation methods on two challenging datasets. The results show that our method can segment the vessels and also detect and track the catheter in the XRA sequences. In general, the results assessed by a cardiologist show that 83% of the images processed by our proposed segmentation method were labeled as good or excellent, while this score for the compared method is 48%. Also, the evaluation results show that our method performs 67% faster than the compared method. Graphical abstract Proposed framework for coronary artery detection.

Multi-focus image fusion using Content Adaptive Blurring

Multi-focus image fusion has emerged as an important research area in information fusion. It aims at increasing the depth-of-field by extracting focused regions from multiple partially focused images, and merging them together to produce a composite image in which all objects are in focus. In this paper, a novel multi-focus image fusion algorithm is presented in which the task of detecting the focused regions is achieved using a Content Adaptive Blurring (CAB) algorithm. The proposed algorithm induces non-uniform blur in a multi-focus image depending on its underlying content. In particular, it analyzes the local image quality in a neighborhood and determines if the blur should be induced or not without losing image quality. In CAB, pixels belonging to the blur regions receive little or no blur at all, whereas the focused regions receive significant blur. Absolute difference of the original image and the CAB-blurred image yields initial segmentation map, which is further refined using morphological operators and graph-cut techniques to improve the segmentation accuracy. Quantitative and qualitative evaluations and comparisons with current state-of-the-art on two publicly available datasets demonstrate the strength of the proposed algorithm.

Patch-Based Segmentation with Spatial Consistency: Application to MS Lesions in Brain MRI.

This paper presents an automatic lesion segmentation method based on similarities between multichannel patches. A patch database is built using training images for which the label maps are known. For each patch in the testing image, k similar patches are retrieved from the database. The matching labels for these k patches are then combined to produce an initial segmentation map for the test case. Finally an iterative patch-based label refinement process based on the initial segmentation map is performed to ensure the spatial consistency of the detected lesions. The method was evaluated in experiments on multiple sclerosis (MS) lesion segmentation in magnetic resonance images (MRI) of the brain. An evaluation was done for each image in the MICCAI 2008 MS lesion segmentation challenge. Results are shown to compete with the state of the art in the challenge. We conclude that the proposed algorithm for segmentation of lesions provides a promising new approach for local segmentation and global detection in medical images.

On the use of binary partition trees for the tree crown segmentation of tropical rainforest hyperspectral images

The segmentation of remotely sensed images acquired over tropical forests is of great interest for numerous ecological applications, such as forest inventories or conservation and management of ecosystems, for which species classification techniques and estimation of the number of individuals are highly valuable inputs. In this paper, we propose a method for hyperspectral image segmentation, based on the binary partition tree (BPT) algorithm, and we apply it to two sites located in Hawaiian and Panamean tropical rainforests. Different strategies combining spatial and spectral dimensionality reduction are compared prior to the construction of the BPT. Various superpixel generation methods including watershed transformation and mean shift clustering are applied to decrease spatial dimensionality and provide an initial segmentation map. Principal component analysis is performed to reduce the spectral dimensionality and different combinations of principal components are compared. A non-parametric region model based on histograms, combined with the diffusion distance to merge regions, is used to build the BPT. An adapted pruning strategy based on the size discontinuity of the merging regions is proposed and compared with an already existing pruning strategy. Finally, a set of criteria to assess the quality of the tree segmentation is introduced. The proposed method correctly segmented up to 68% of the tree crowns and produced reasonable patterns of the segmented landscapes.

Self-supervised texture segmentation using complementary types of features

A two-stage texture segmentation approach is proposed where an initial segmentation map is obtained through unsupervised clustering of multiresolution simultaneous autoregressive (MRSAR) features and is followed by self-supervised classification of wavelet features. The regions of “high confidence” and “low confidence” are identified based on the MRSAR segmentation result using multilevel morphological erosion. The second-stage classifier is trained by the “high-confidence” samples and is used to reclassify only the “low-confidence” pixels. The proposed approach leverages on the advantages of both MRSAR and wavelet features. Experimental results show that the misclassification error can be significantly reduced by using complementary types of texture features.