Medical Segmentation Methods Research Articles

Anatomically plausible segmentations: Explicitly preserving topology through prior deformations

Since the rise of deep learning, new medical segmentation methods have rapidly been proposed with extremely promising results, often reporting marginal improvements on the previous state-of-the-art (SOTA) method. However, on visual inspection errors are often revealed, such as topological mistakes (e.g. holes or folds), that are not detected using traditional evaluation metrics. Incorrect topology can often lead to errors in clinically required downstream image processing tasks. Therefore, there is a need for new methods to focus on ensuring segmentations are topologically correct. In this work, we present TEDS-Net: a segmentation network that preserves anatomical topology whilst maintaining segmentation performance that is competitive with SOTA baselines. Further, we show how current SOTA segmentation methods can introduce problematic topological errors. TEDS-Net achieves anatomically plausible segmentation by using learnt topology-preserving fields to deform a prior. Traditionally, topology-preserving fields are described in the continuous domain and begin to break down when working in the discrete domain. Here, we introduce additional modifications that more strictly enforce topology preservation. We illustrate our method on an open-source medical heart dataset, performing both single and multi-structure segmentation, and show that the generated fields contain no folding voxels, which corresponds to full topology preservation on individual structures whilst vastly outperforming the other baselines on overall scene topology. The code is available at: https://github.com/mwyburd/TEDS-Net.

Multiple organ segmentation framework for brain metastasis radiotherapy

Radiotherapy is a preferred treatment for brain metastases, which kills cancer cells via high doses of radiation meanwhile hardly avoiding damage to surrounding healthy cells. Therefore, the delineation of organs-at-risk (OARs) is vital in treatment planning to minimize radiation-induced toxicity. However, the following aspects make OAR delineation a challenging task: extremely imbalanced organ sizes, ambiguous boundaries, and complex anatomical structures. To alleviate these challenges, we imitate how specialized clinicians delineate OARs and present a novel cascaded multi-OAR segmentation framework, called OAR-SegNet. OAR-SegNet comprises two distinct levels of segmentation networks: an Anatomical-Prior-Guided network (APG-Net) and a Point-Cloud-Guided network (PCG-Net). Specifically, APG-Net handles segmentation for all organs, where multi-view segmentation modules and a deep prior loss are designed under the guidance of prior knowledge. After APG-Net, PCG-Net refines small organs through the mini-segmentation and the point-cloud alignment heads. The mini-segmentation head is further equipped with the deep prior feature. Extensive experiments were conducted to demonstrate the superior performance of the proposed method compared to other state-of-the-art medical segmentation methods.

SEA-NET: medical image segmentation network based on spiral squeeze-and-excitation and attention modules

BackgroundMedical image segmentation is an important processing step in most of medical image analysis. Thus, high accuracy and robustness are required for them. The current deep neural network based medical segmentation methods have good effect on image with balanced foreground and background, but it will loss the characteristics of small targets on image with imbalanced foreground and background after multiple convolutions.MethodsIn order to retain the features of small targets in the deep network, we proposed a new medical image segmentation model based on the U-Net with squeeze-and-excitation and attention modules which form a spiral closed path,callled as Spiral Squeeze-and-Excitation and Attention NET (SEA-NET) in this paper. The segmentation model used squeeze-and-extraction modules to adjust the channel information to enhance the useful information and used attention modules to adjust the spatial information of the feature map to highlight the target area for small target segmentation when up-sampling. The deep semantic information is integrated into the shallow feature map by the attention model. Therefore, the deep semantic information cannot be scattered by continuous up-sampling. We used cross entropy loss + Tversky loss function for fast convergence and well processing the imbalanced data sets. Our proposed SEA-NET was tested on the brain MRI dataset LPBA40 and peripheral blood smear images.ConclusionsOn brain MRI data, the average value of the Dice coefficient we obtained reached 98.1%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document}. On the peripheral blood smear dataset, our proposed model has a good segmentation effect on adhesion cells.ResultsThe experimental results proved that the proposed SEA-Net performed better than U-Net, U-Net++, etc. in medical image segmentation.

Learning active contour models based on self-attention for breast ultrasound image segmentation

Computer-aided diagnosis (CAD) systems based on ultrasound have been developed and widely promoted in breast cancer screening. Due to the characteristics of low contrast and speckle noises, breast ultrasound image segmentation, one of the crucial steps of CAD systems, has always been challenging. Recently, the emerging Transformer-based medical segmentation methods, which have a better ability to model long dependencies than convolutional neural networks (CNNs), have shown significant value for medical image segmentation. However, due to the limited data with the high-quality label, Transformer performs weakly on breast ultrasound image segmentation without pretraining. Thus, we propose the Attention-Gate Medical Transformer (AGMT) for small breast ultrasound datasets, which introduces the attention-gate (AG) module to suppress background information and the average radial derivative increment (ΔARD) loss function to enhance shape information. We evaluate the AGMT on both a private dataset A and a public dataset B. On dataset A, the AGMT outperforms MT on the metrics of true positive ratio, jaccard index (JI) and dice similarity coefficient (DSC) by 6.4%, 2.3% and 1.9%, respectively. Meanwhile, when compared with UNet, the AGMT improves JI and DSC by 5.3% and 4.9%, respectively. The results show performance has significantly improved compared with mainstream models. In addition, we also conduct ablation experiments on the AG module and ΔARD, which prove their effectiveness.

Pixel Diffuser: Practical Interactive Medical Image Segmentation without Ground Truth.

Medical image segmentation is essential for doctors to diagnose diseases and manage patient status. While deep learning has demonstrated potential in addressing segmentation challenges within the medical domain, obtaining a substantial amount of data with accurate ground truth for training high-performance segmentation models is both time-consuming and demands careful attention. While interactive segmentation methods can reduce the costs of acquiring segmentation labels for training supervised models, they often still necessitate considerable amounts of ground truth data. Moreover, achieving precise segmentation during the refinement phase results in increased interactions. In this work, we propose an interactive medical segmentation method called PixelDiffuser that requires no medical segmentation ground truth data and only a few clicks to obtain high-quality segmentation using a VGG19-based autoencoder. As the name suggests, PixelDiffuser starts with a small area upon the initial click and gradually detects the target segmentation region. Specifically, we segment the image by creating a distortion in the image and repeating it during the process of encoding and decoding the image through an autoencoder. Consequently, PixelDiffuser enables the user to click a part of the organ they wish to segment, allowing the segmented region to expand to nearby areas with pixel values similar to the chosen organ. To evaluate the performance of PixelDiffuser, we employed the dice score, based on the number of clicks, to compare the ground truth image with the inferred segment. For validation of our method's performance, we leveraged the BTCV dataset, containing CT images of various organs, and the CHAOS dataset, which encompasses both CT and MRI images of the liver, kidneys and spleen. Our proposed model is an efficient and effective tool for medical image segmentation, achieving competitive performance compared to previous work in less than five clicks and with very low memory consumption without additional training.

TANet: Triple Attention Network for medical image segmentation

In recent years, deep learning-based methods have achieved remarkable progress in medical image processing, like polyp segmentation in colonoscopy images and skin lesion segmentation in dermoscopy images. However, the current state-of-the-art medical segmentation methods still suffer from the problem of low accuracy in segmenting the small-scale and variable-scale objects. To solve this problem, we propose Triple Attention Network (TANet). In TANet, a novel Triple Attention Module (TAM) is presented. TAM has two sub-modules: Multi-scale Feature Selection Module (MFSM) and Contextual Feature Extraction Module (CFEM). MFSM is used to extract more adaptable multi-scale features for capturing variable-scale objects, while CFEM is for capturing small-scale objects by extracting contextual features. TAM aims to combine MFSM and CFEM to finally enhance the segmentation performance of the medical images with the small-scale and variable-scale lesions. Extensive experiments are conducted on five polyp datasets and one skin lesion dataset. Results show that the proposed models outperform the previous state-of-the-art models on most evaluation metrics and improve the Dice score by up to 7.1%. All results consistently confirm the effectiveness of the proposed TANet and show that the TANet achieves state-of-the-art performance on the above datasets.

Implicit Anatomical Rendering for Medical Image Segmentation with Stochastic Experts.

Integrating high-level semantically correlated contents and low-level anatomical features is of central importance in medical image segmentation. Towards this end, recent deep learning-based medical segmentation methods have shown great promise in better modeling such information. However, convolution operators for medical segmentation typically operate on regular grids, which inherently blur the high-frequency regions, i.e., boundary regions. In this work, we propose MORSE, a generic implicit neural rendering framework designed at an anatomical level to assist learning in medical image segmentation. Our method is motivated by the fact that implicit neural representation has been shown to be more effective in fitting complex signals and solving computer graphics problems than discrete grid-based representation. The core of our approach is to formulate medical image segmentation as a rendering problem in an end-to-end manner. Specifically, we continuously align the coarse segmentation prediction with the ambiguous coordinate-based point representations and aggregate these features to adaptively refine the boundary region. To parallelly optimize multi-scale pixel-level features, we leverage the idea from Mixture-of-Expert (MoE) to design and train our MORSE with a stochastic gating mechanism. Our experiments demonstrate that MORSE can work well with different medical segmentation backbones, consistently achieving competitive performance improvements in both 2D and 3D supervised medical segmentation methods. We also theoretically analyze the superiority of MORSE.

Self-Supervised Multi-Modal Hybrid Fusion Network for Brain Tumor Segmentation

Accurate medical image segmentation of brain tumors is necessary for the diagnosing, monitoring, and treating disease. In recent years, with the gradual emergence of multi-sequence magnetic resonance imaging (MRI), multi-modal MRI diagnosis has played an increasingly important role in the early diagnosis of brain tumors by providing complementary information for a given lesion. Different MRI modalities vary significantly in context, as well as in coarse and fine information. As the manual identification of brain tumors is very complicated, it usually requires the lengthy consultation of multiple experts. The automatic segmentation of brain tumors from MRI images can thus greatly reduce the workload of doctors and buy more time for treating patients. In this paper, we propose a multi-modal brain tumor segmentation framework that adopts the hybrid fusion of modality-specific features using a self-supervised learning strategy. The algorithm is based on a fully convolutional neural network. Firstly, we propose a multi-input architecture that learns independent features from multi-modal data, and can be adapted to different numbers of multi-modal inputs. Compared with single-modal multi-channel networks, our model provides a better feature extractor for segmentation tasks, which learns cross-modal information from multi-modal data. Secondly, we propose a new feature fusion scheme, named hybrid attentional fusion. This scheme enables the network to learn the hybrid representation of multiple features and capture the correlation information between them through an attention mechanism. Unlike popular methods, such as feature map concatenation, this scheme focuses on the complementarity between multi-modal data, which can significantly improve the segmentation results of specific regions. Thirdly, we propose a self-supervised learning strategy for brain tumor segmentation tasks. Our experimental results demonstrate the effectiveness of the proposed model against other state-of-the-art multi-modal medical segmentation methods.

Application of CT Multimodal Images in Rehabilitation Monitoring of Long-Distance Running.

In order to monitor the rehabilitation of athletes injured in long-distance running, the author proposes a method for rehabilitation monitoring of long-distance running based on CT multimodal images. This method combines the latest multimodal image technology, integrates multimodal technology into CT images to improve the accuracy, performs image segmentation on CT multimodal images through medical segmentation methods, and analyzes the segmented images; finally, it can achieve the effect of rehabilitation treatment for athletes in long-distance running. Experimental results show that the total time taken by the authors' method is 10.9 hours, with an average time of 8 seconds, which is much shorter than the other two control methods. In conclusion, the authors' method allows for better rehabilitation monitoring of long-distance running sports injuries.

Modeling the Probabilistic Distribution of Unlabeled Data for One-shot Medical Image Segmentation

Existing image segmentation networks mainly leverage large-scale labeled datasets to attain high accuracy. However, labeling medical images is very expensive since it requires sophisticated expert knowledge. Thus, it is more desirable to employ only a few labeled data in pursuing high segmentation performance. In this paper, we develop a data augmentation method for one-shot brain magnetic resonance imaging (MRI) image segmentation which exploits only one labeled MRI image (named atlas) and a few unlabeled images. In particular, we propose to learn the probability distributions of deformations (including shapes and intensities) of different unlabeled MRI images with respect to the atlas via 3D variational autoencoders (VAEs). In this manner, our method is able to exploit the learned distributions of image deformations to generate new authentic brain MRI images, and the number of generated samples will be sufficient to train a deep segmentation network. Furthermore, we introduce a new standard segmentation benchmark to evaluate the generalization performance of a segmentation network through a cross-dataset setting (collected from different sources). Extensive experiments demonstrate that our method outperforms the state-of-the-art one-shot medical segmentation methods. Our code has been released at https://github.com/dyh127/Modeling-the-Probabilistic-Distribution-of-Unlabeled-Data.

Medical image segmentation algorithm based on multilayer boundary perception-self attention deep learning model

Traditional medical image segmentation methods have problems such as low segmentation accuracy and low adaptive ability. Therefore, many scholars have proposed a medical image segmentation method based on deep learning, which has achieved good results in the field of medical image segmentation. However, this type of method has the following problems in the application process: (1) Medical image segmentation target boundary positioning problem. Constrained by factors such as medical image contrast, heterogeneity, and boundary resolution, existing convolution models still cannot accurately locate boundaries. (2) Deep adaptability of deep learning network structure to medical images. Because medical images have more distinct and different feature information than natural images, the current deep learning-based medical segmentation methods have not fully considered this feature. In view of this, this paper proposes a multi-level boundary-aware RUNet segmentation model. The network structure consists of a U-Net-based segmentation network and a multi-level boundary detection network. It can solve the problem of boundary positioning. At the same time, in order to solve the problem of poor adaptability of deep learning network structures to medical images, this paper proposes to introduce a new interactive self-attention module into deep learning models. It can make the feature map get global information, and realize the effective extraction of medical image feature information. It solves the problem of weak matching between the deep learning network structure and medical images. Based on the above ideas, this paper proposes an image segmentation algorithm based on a multi-layer boundary perception-self-attention mechanism deep learning model. This method and other mainstream segmentation algorithms are used to perform experiments on related medical databases. The results show that the proposed method not only improves the segmentation effect significantly compared with traditional machine learning methods, but also improves it to a certain extent compared with other deep learning methods.

Multi-modal medical image segmentation based on vector-valued active contour models

Positron emission tomography (PET), magnetic resonance imaging (MRI) and computed tomography (CT) are widely utilized medical imaging modalities that provide essential anatomic and structural details. Many medical segmentation methods are not effective for a single-modal image of poor quality (e.g., low contrast in CT or low spatial resolution in PET). For practical radiotherapy treatment planning, multi-modal imaging information is regularly used. In this paper, a novel vector-valued active contour model is proposed to segment multi-modal medical images simultaneously for abnormal tissue regions. The method makes use of the functionality information and anatomical structure information advantages from each modality. Since each modality has its own signal characteristics, we use region-based information, combining hybrid mean intensities simultaneously. Furthermore, by utilizing a two-dimensional vector field with different image modalities, edge-based information is used to constrain the results of the image segmentation. The proposed approach is evaluated on datasets including lung PET-CT and brain MRI-CT images. Our qualitative and quantitative research results confirm the effectiveness of the proposed method.

Cascaded atrous convolution and spatial pyramid pooling for more accurate tumor target segmentation for rectal cancer radiotherapy

Convolutional neural networks (CNNs) have become the state-of-the-art method for medical segmentation. However, repeated pooling and striding operations reduce the feature resolution, causing loss of detailed information. Additionally, tumors of different patients are of different sizes. Thus, small tumors may be ignored while big tumors may exceed the receptive fields of convolutions. The purpose of this study is to further improve the segmentation accuracy using a novel CNN (named CAC–SPP) with cascaded atrous convolution (CAC) and a spatial pyramid pooling (SPP) module. This work is the first attempt at applying SPP for segmentation in radiotherapy. We improved the network based on ResNet-101 yielding accuracy gains from a greatly increased depth. We added CAC to extract a high-resolution feature map while maintaining large receptive fields. We also adopted a parallel SPP module with different atrous rates to capture the multi-scale features. The performance was compared with the widely adopted U-Net and ResNet-101 with independent segmentation of rectal tumors for two image sets, separately: (1) 70 T2-weighted MR images and (2) 100 planning CT images. The results show that the proposed CAC–SPP outperformed the U-Net and ResNet-101 for both image sets. The Dice similarity coefficient values of CAC–SPP were 0.78 ± 0.08 and 0.85 ± 0.03, respectively, which were higher than those of U-Net (0.70 ± 0.11 and 0.82 ± 0.04) and ResNet-101 (0.76 ± 0.10 and 0.84 ± 0.03). The segmentation speed of CAC–SPP was comparable with ResNet-101, but about 36% faster than U-Net. In conclusion, the proposed CAC–SPP, which could extract high-resolution features with large receptive fields and capture multi-scale context yields, improves the accuracy of segmentation performance for rectal tumors.

Reformulating Level Sets as Deep Recurrent Neural Network Approach to Semantic Segmentation.

Variational Level Set (LS) has been a widely used method in medical segmentation. However, it is limited when dealing with multi-instance objects in the real world. In addition, its segmentation results are quite sensitive to initial settings and highly depend on the number of iterations. To address these issues and boost the classic variational LS methods to a new level of the learnable deep learning approaches, we propose a novel definition of contour evolution named Recurrent Level Set (RLS) 1 to employ Gated Recurrent Unit under the energy minimization of a variational LS functional. The curve deformation process in RLS is formed as a hidden state evolution procedure and updated by minimizing an energy functional composed of fitting forces and contour length. By sharing the convolutional features in a fully end-to-end trainable framework, we extend RLS to Contextual RLS (CRLS) to address semantic segmentation in the wild. The experimental results have shown that our proposed RLS improves both computational time and segmentation accuracy against the classic variational LS-based method whereas the fully end-to-end system CRLS achieves competitive performance compared to the state-of-the-art semantic segmentation approaches.

A Survey of Graph Cuts/Graph Search Based Medical Image Segmentation.

Medical image segmentation is a fundamental and challenging problem for analyzing medical images. Among different existing medical image segmentation methods, graph-based approaches are relatively new and show good features in clinical applications. In the graph-based method, pixels or regions in the original image are interpreted into nodes in a graph. By considering Markov random field to model the contexture information of the image, the medical image segmentation problem can be transformed into a graph-based energy minimization problem. This problem can be solved by the use of minimum s-t cut/ maximum flow algorithm. This review is devoted to cut-based medical segmentation methods, including graph cuts and graph search for region and surface segmentation. Different varieties of cut-based methods, including graph-cuts-based methods, model integrated graph cuts methods, graph-search-based methods, and graph search/graph cuts based methods, are systematically reviewed. Graph cuts and graph search with deep learning technique are also discussed.

A new multistage medical segmentation method based on superpixel and fuzzy clustering.

The medical image segmentation is the key approach of image processing for brain MRI images. However, due to the visual complex appearance of image structures and the imaging characteristic, it is still challenging to automatically segment brain MRI image. A new multi-stage segmentation method based on superpixel and fuzzy clustering (MSFCM) is proposed to achieve the good brain MRI segmentation results. The MSFCM utilizes the superpixels as the clustering objects instead of pixels, and it can increase the clustering granularity and overcome the influence of noise and bias effectively. In the first stage, the MRI image is parsed into several atomic areas, namely, superpixels, and a further parsing step is adopted for the areas with bigger gray variance over setting threshold. Subsequently, designed fuzzy clustering is carried out to the fuzzy membership of each superpixel, and an iterative broadcast method based on the Butterworth function is used to redefine their classifications. Finally, the segmented image is achieved by merging the superpixels which have the same classification label. The simulated brain database from BrainWeb site is used in the experiments, and the experimental results demonstrate that MSFCM method outperforms the traditional FCM algorithm in terms of segmentation accuracy and stability for MRI image.

A New Medical Segmentation Method Based on Active Contour Models and Roberts

A New Medical Segmentation Method Based on Active Contour Models and Roberts