High Segmentation Performance Research Articles

Enhanced multistage deep learning for diagnosing anterior disc displacement in temporomandibular joint using magnetic resonance imaging.

This study aimed to propose a new method for the automatic diagnosis of anterior disc displacement of the temporomandibular joint (TMJ) using magnetic resonance imaging (MRI) and deep learning. By employing a multistage approach, the factors affecting the final result can be easily identified and improved. This study introduces a multistage automatic diagnostic technique using deep learning. This process involves segmenting the target from MR images, extracting distance parameters, and classifying the diagnosis into three classes. MRI exams of 368 TMJs from 204 patients were evaluated for anterior disc displacement. In the first stage, five algorithms were used for the semantic segmentation of the disc and condyle. In the second stage, 54 distance parameters were extracted from the segments. In the third stage, a rule-based decision model was developed to link the parameters with the expert diagnosis results. In the first stage, DeepLabV3+ showed the best result (95% Hausdorff distance, Dice coefficient, and sensitivity of 6.47 ± 7.22, 0.84 ± 0.07, and 0.84 ± 0.09, respectively). This study used the original MRI exams as input without preprocessing and showed high segmentation performance compared with that of previous studies. In the third stage, the combination of SegNet and a random forest model yielded an accuracy of 0.89 ± 0.06. An algorithm was developed to automatically diagnose TMJ-anterior disc displacement using MRI. Through a multistage approach, this algorithm facilitated the improvement of results and demonstrated high accuracy from more complex inputs. Furthermore, existing radiological knowledge was applied and validated.

Novel Domain Knowledge-Encoding Algorithm Enables Label-Efficient Deep Learning for Cardiac CT Segmentation to Guide Atrial Fibrillation Treatment in a Pilot Dataset.

Background: Segmenting computed tomography (CT) is crucial in various clinical applications, such as tailoring personalized cardiac ablation for managing cardiac arrhythmias. Automating segmentation through machine learning (ML) is hindered by the necessity for large, labeled training data, which can be challenging to obtain. This article proposes a novel approach for automated, robust labeling using domain knowledge to achieve high-performance segmentation by ML from a small training set. The approach, the domain knowledge-encoding (DOKEN) algorithm, reduces the reliance on large training datasets by encoding cardiac geometry while automatically labeling the training set. The method was validated in a hold-out dataset of CT results from an atrial fibrillation (AF) ablation study. Methods: The DOKEN algorithm parses left atrial (LA) structures, extracts "anatomical knowledge" by leveraging digital LA models (available publicly), and then applies this knowledge to achieve high ML segmentation performance with a small number of training samples. The DOKEN-labeled training set was used to train a nnU-Net deep neural network (DNN) model for segmenting cardiac CT in N = 20 patients. Subsequently, the method was tested in a hold-out set with N = 100 patients (five times larger than training set) who underwent AF ablation. Results: The DOKEN algorithm integrated with the nn-Unet model achieved high segmentation performance with few training samples, with a training to test ratio of 1:5. The Dice score of the DOKEN-enhanced model was 96.7% (IQR: 95.3% to 97.7%), with a median error in surface distance of boundaries of 1.51 mm (IQR: 0.72 to 3.12) and a mean centroid-boundary distance of 1.16 mm (95% CI: -4.57 to 6.89), similar to expert results (r = 0.99; p < 0.001). In digital hearts, the novel DOKEN approach segmented the LA structures with a mean difference for the centroid-boundary distances of -0.27 mm (95% CI: -3.87 to 3.33; r = 0.99; p < 0.0001). Conclusions: The proposed novel domain knowledge-encoding algorithm was able to perform the segmentation of six substructures of the LA, reducing the need for large training data sets. The combination of domain knowledge encoding and a machine learning approach could reduce the dependence of ML on large training datasets and could potentially be applied to AF ablation procedures and extended in the future to other imaging, 3D printing, and data science applications.

PCa-RadHop: A transparent and lightweight feed-forward method for clinically significant prostate cancer segmentation

Prostate Cancer is one of the most frequently occurring cancers in men, with a low survival rate if not early diagnosed. PI-RADS reading has a high false positive rate, thus increasing the diagnostic incurred costs and patient discomfort. Deep learning (DL) models achieve a high segmentation performance, although require a large model size and complexity. Also, DL models lack of feature interpretability and are perceived as “black-boxes” in the medical field. PCa-RadHop pipeline is proposed in this work, aiming to provide a more transparent feature extraction process using a linear model. It adopts the recently introduced Green Learning (GL) paradigm, which offers a small model size and low complexity. PCa-RadHop consists of two stages: Stage-1 extracts data-driven radiomics features from the bi-parametric Magnetic Resonance Imaging (bp-MRI) input and predicts an initial heatmap. To reduce the false positive rate, a subsequent stage-2 is introduced to refine the predictions by including more contextual information and radiomics features from each already detected Region of Interest (ROI). Experiments on the largest publicly available dataset, PI-CAI, show a competitive performance standing of the proposed method among other deep DL models, achieving an area under the curve (AUC) of 0.807 among a cohort of 1,000 patients. Moreover, PCa-RadHop maintains orders of magnitude smaller model size and complexity.

RailSegVITNet: A lightweight VIT-based real-time track surface segmentation network for improving railroad safety

To address the challenge of precise foreign object detection on railway tracks, this study proposes RailSegVITNet, an efficient deep learning-based model that aims to accurately locate tracks and provide necessary information. RailSegVITNet integrates lightweight bottleneck blocks, separable self-attention, and feature aggregation to balance real-time performance and accuracy. It follows an encoder–decoder framework, with bottleneck blocks used for feature extraction and separable self-attention integrated at different stages to enhance feature extraction capability. The decoder utilizes feature aggregation to effectively merge multi-resolution features and improve segmentation performance. On the proposed Railway-seg dataset, the RailSegVITNet demonstrates competitive performance, achieving an MIoU (Mean Intersection over Union ) of 91.43% while utilizing only 2.01 G FLOPs (Floating Point Operations) and 1.4 M parameters. Furthermore, compared to popular models such as Topformer, Segformer, Segmenter, and Deeplab on the publicly available Railsem19 dataset, RailSegVITNet maintains its lightweight architecture while achieving comparable or higher segmentation performance. This showcases its potential practical value for efficient identification of foreign object intrusion.

LST-AI: A deep learning ensemble for accurate MS lesion segmentation

Automated segmentation of brain white matter lesions is crucial for both clinical assessment and scientific research in multiple sclerosis (MS). Over a decade ago, we introduced an engineered lesion segmentation tool, LST. While recent lesion segmentation approaches have leveraged artificial intelligence (AI), they often remain proprietary and difficult to adopt. As an open-source tool, we present LST-AI, an advanced deep learning-based extension of LST that consists of an ensemble of three 3D U-Nets.LST-AI explicitly addresses the imbalance between white matter (WM) lesions and non-lesioned WM. It employs a composite loss function incorporating binary cross-entropy and Tversky loss to improve segmentation of the highly heterogeneous MS lesions. We train the network ensemble on 491 MS pairs of T1-weighted and FLAIR images, collected in-house from a 3T MRI scanner, and expert neuroradiologists manually segmented the utilized lesion maps for training. LST-AI also includes a lesion location annotation tool, labeling lesions as periventricular, infratentorial, and juxtacortical according to the 2017 McDonald criteria, and, additionally, as subcortical. We conduct evaluations on 103 test cases consisting of publicly available data using the Anima segmentation validation tools and compare LST-AI with several publicly available lesion segmentation models.Our empirical analysis shows that LST-AI achieves superior performance compared to existing methods. Its Dice and F1 scores exceeded 0.62, outperforming LST, SAMSEG (Sequence Adaptive Multimodal SEGmentation), and the popular nnUNet framework, which all scored below 0.56. Notably, LST-AI demonstrated exceptional performance on the MSSEG-1 challenge dataset, an international WM lesion segmentation challenge, with a Dice score of 0.65 and an F1 score of 0.63—surpassing all other competing models at the time of the challenge. With increasing lesion volume, the lesion detection rate rapidly increased with a detection rate of >75% for lesions with a volume between 10 mm3 and 100 mm3. Given its higher segmentation performance, we recommend that research groups currently using LST transition to LST-AI. To facilitate broad adoption, we are releasing LST-AI as an open-source model, available as a command-line tool, dockerized container, or Python script, enabling diverse applications across multiple platforms.

Weakly supervised image segmentation beyond tight bounding box annotations

Weakly supervised image segmentation approaches in the literature usually achieve high segmentation performance using tight bounding box supervision and decrease the performance greatly when supervised by loose bounding boxes. However, compared with loose bounding box, it is much more difficult to acquire tight bounding box due to its strict requirements on the precise locations of the four sides of the box. To resolve this issue, this study investigates whether it is possible to maintain good segmentation performance when loose bounding boxes are used as supervision. For this purpose, this work extends our previous parallel transformation based multiple instance learning (MIL) for tight bounding box supervision by integrating an MIL strategy based on polar transformation to assist image segmentation. The proposed polar transformation based MIL formulation works for both tight and loose bounding boxes, in which a positive bag is defined as pixels in a polar line of a bounding box with one endpoint located inside the object enclosed by the box and the other endpoint located at one of the four sides of the box. Moreover, a weighted smooth maximum approximation is introduced to incorporate the observation that pixels closer to the origin of the polar transformation are more likely to belong to the object in the box. The proposed approach was evaluated on two public datasets using dice coefficient when bounding boxes at different precision levels were considered in the experiments. The results demonstrate that the proposed approach achieves state-of-the-art performance for bounding boxes at all precision levels and is robust to mild and moderate errors in the loose bounding box annotations. The codes are available at https://github.com/wangjuan313/wsis-beyond-tightBB.

Your Input Matters—Comparing Real-Valued PolSAR Data Representations for CNN-Based Segmentation

Inspired by the success of Convolutional Neural Network (CNN)-based deep learning methods for optical image segmentation, there is a growing interest in applying these methods to Polarimetric Synthetic Aperture Radar (PolSAR) data. However, effectively utilizing well-established real-valued CNNs for PolSAR image segmentation requires converting complex-valued data into real-valued representations. This paper presents a systematic comparison of 14 different real-valued representations used as CNN input in the literature. These representations encompass various approaches, including the use of coherency matrix elements, hand-crafted feature vectors, polarimetric features based on target decomposition, and combinations of these methods. The goal is to assess the impact of the choice of PolSAR data representation on segmentation performance and identify the most suitable representation. Four test configurations are employed to achieve this, involving different CNN architectures (U-Net with ResNet-18 or EfficientNet backbone) and PolSAR data acquired in different frequency bands (S- and L-band). The results emphasize the importance of selecting an appropriate real-valued representation for CNN-based PolSAR image segmentation. This study’s findings reveal that combining multiple polarimetric features can potentially enhance segmentation performance but does not consistently improve the results. Therefore, when employing this approach, careful feature selection becomes crucial. In contrast, using coherency matrix elements with amplitude and phase representation consistently achieves high segmentation performance across different test configurations. This representation emerges as one of the most suitable approaches for CNN-based PolSAR image segmentation. Notably, it outperforms the commonly used alternative approach of splitting the coherency matrix elements into real and imaginary parts.

Evolutionary Architecture Optimization for Retinal Vessel Segmentation.

Retinal vessel segmentation (RVS) is crucial in medical image analysis as it helps identify and monitor retinal diseases. Deep learning approaches have shown promising results for RVS, but designing optimal neural network architecture is challenging and time-consuming. Neural architecture search (NAS) is a recent technique that automates the design of neural network architectures within a predefined search space. This study proposes a new NAS method for U-shaped networks, MedUNAS, that discovers deep neural networks with high segmentation performance and lower inference time for RVS problem. We perform opposition-based differential evolution (ODE) and genetic algorithm (GA) to search for the best network structure and compare discrete and continuous encoding strategies on the proposed search space. To the best of our knowledge, this is the first NAS study that performs ODE for RVS problems. The results show that the MedUNAS ODE and GA yield the best and second-best results regarding segmentation performance with less than 50% of the parameters of U-shaped state-of-the-art methods on most of the compared datasets. In addition, the proposed methods outperform the baseline U-Net on four datasets with networks with up to 15 times fewer parameters. Furthermore, ablation studies are performed to evaluate the generalizability of the generated networks to medical image segmentation problems that differ from the trained domain, revealing that such networks can be effectively adapted to new tasks with fine-tuning. The MedUNAS can be a valuable tool for automated and efficient RVS in clinical practice.

FastSurfer-HypVINN: Automated sub-segmentation of the hypothalamus and adjacent structures on high-resolutional brain MRI

Abstract The hypothalamus plays a crucial role in the regulation of a broad range of physiological, behavioral, and cognitive functions. However, despite its importance, only a few small-scale neuroimaging studies have investigated its substructures, likely due to the lack of fully automated segmentation tools to address scalability and reproducibility issues of manual segmentation. While the only previous attempt to automatically sub-segment the hypothalamus with a neural network showed promise for 1.0 mm isotropic T1-weighted (T1w) magnetic resonance imaging (MRI), there is a need for an automated tool to sub-segment also high-resolutional (HiRes) MR scans, as they are becoming widely available, and include structural detail also from multi-modal MRI. We, therefore, introduce a novel, fast, and fully automated deep-learning method named HypVINN for sub-segmentation of the hypothalamus and adjacent structures on 0.8 mm isotropic T1w and T2w brain MR images that is robust to missing modalities. We extensively validate our model with respect to segmentation accuracy, generalizability, in-session test-retest reliability, and sensitivity to replicate hypothalamic volume effects (e.g., sex differences). The proposed method exhibits high segmentation performance both for standalone T1w images as well as for T1w/T2w image pairs. Even with the additional capability to accept flexible inputs, our model matches or exceeds the performance of state-of-the-art methods with fixed inputs. We, further, demonstrate the generalizability of our method in experiments with 1.0 mm MR scans from both the Rhineland Study and the UK Biobank—an independent dataset never encountered during training with different acquisition parameters and demographics. Finally, HypVINN can perform the segmentation in less than a minute (graphical processing unit [GPU]) and will be available in the open source FastSurfer neuroimaging software suite, offering a validated, efficient, and scalable solution for evaluating imaging-derived phenotypes of the hypothalamus.

CSwinDoubleU-Net: A double U-shaped network combined with convolution and Swin Transformer for colorectal polyp segmentation

The early detection of polyps from colonoscopy can reduce the risk of colorectal cancer development and give timely treatment. Accurate polyp segmentation during colonoscopy examinations can help clinicians locate the polyps which is of great significance in the clinical prevention of colorectal cancer. However, due to large variations in terms of size, color, texture, and morphology of polyps, the similarity between the polyp lesions and their background, the variations of illumination, motion blur, low-contrast areas, intestinal contents during image acquisitions, precisely polyp segmentation is still an open issue. To overcome the challenges above, a new double U-shaped image segmentation network combining convolution structure and Shifted Windows (Swin) Transformer, namely CSwinDoubleU-Net, is proposed in this paper. The developed CSwinDoubleU-Net consists of two U-shaped encode-decode structures associated with a pure CNN-based structure and a pure Transformer-based one, and a feature fusion module that interacts with the two U-shaped parts. The first U-shaped codecs network utilizes the multi-convolution layers to extract the local feature information and the coordinate attention module in each skip connection to effectively reduce the loss of spatial information and obtain the accurate position information of the encoded features. Next, the second U-shaped codecs network uses the Swin Transformer layers with the sliding window to extract global feature information further. At last, a convolutional feature and self-attention feature fusion module (CSFFM) is designed to deeply fuse the local convolutional features extracted from the first U-shape structure and the global self-Attention features extracted from the second U-shape structure. The obtained multi-category and multi-dimensional fused feature information can help to recover the boundary features of polyps. Extensive experiments are conducted to validate the proposed CSwinDoubleU-Net on five publicly available datasets, including CVC-ClinicDB, Kvasir, CVC-ColonDB, CVC-T, and ETIS-Larib. The results show that the proposed model can outperform some state-of-the-art methods and achieve high segmentation performance for polyp images.

Correlating Deep Learning-Based Automated Reference Kidney Histomorphometry with Patient Demographics and Creatinine.

Reference histomorphometric data of healthy human kidneys are largely lacking due to laborious quantitation requirements. Correlating histomorphometric features with clinical parameters through machine learning approaches can provide valuable information about natural population variance. To this end, we leveraged deep learning, computational image analysis, and feature analysis to associate the relationship of histomorphometry with patient age, sex, serum creatinine (SCr), and estimated glomerular filtration rate (eGFR) in a multinational set of reference kidney tissue sections. A panoptic segmentation neural network was developed and used to segment viable and sclerotic glomeruli, cortical and medullary interstitia, tubules, and arteries/arterioles in the digitized images of 79 periodic acid-Schiff-stained human nephrectomy sections showing minimal pathologic changes. Simple morphometrics (e.g., area, radius, density) were quantified from the segmented classes. Regression analysis aided in determining the association of histomorphometric parameters with age, sex, SCr, and eGFR. Our deep-learning model achieved high segmentation performance for all test compartments. The size and density of glomeruli, tubules, and arteries/arterioles varied significantly among healthy humans, with potentially large differences between geographically diverse patients. Glomerular size was significantly correlated with SCr and eGFR. Slight, albeit significant, differences in renal vasculature were observed between sexes. Glomerulosclerosis percentage increased and cortical density of arteries/arterioles decreased, as a function of increasing age. Using deep learning, we automated precise measurements of kidney histomorphometric features. In the reference kidney tissue, several histomorphometric features demonstrated significant correlation to patient demographics, SCr, and eGFR. Deep learning tools can increase the efficiency and rigor of histomorphometric analysis.

Adaptive Decomposition and Shared Weight Volumetric Transformer Blocks for Efficient Patch-free 3D Medical Image Segmentation.

High resolution (HR) 3D medical image segmentation is vital for an accurate diagnosis. However, in the field of medical imaging, it is still a challenging task to achieve a high segmentation performance with cost-effective and feasible computation resources. Previous methods commonly use patch-sampling to reduce the input size, but this inevitably harms the global context and decreases the model's performance. In recent years, a few patch-free strategies have been presented to deal with this issue, but either they have limited performance due to their over-simplified model structures or they follow a complicated training process. In this study, to effectively address these issues, we present Adaptive Decomposition (A-Decomp) and Shared Weight Volumetric Transformer Blocks (SW-VTB). A-Decomp can adaptively decompose features and reduce their spatial size, which greatly lowers GPU memory consumption. SW-VTB is able to capture long-range dependencies at a low cost with its lightweight design and cross-scale weight-sharing mechanism. Our proposed cross-scale weight-sharing approach enhances the network's ability to capture scale-invariant core semantic information in addition to reducing parameter numbers. By combining these two designs together, we present a novel patch-free segmentation framework named VolumeFormer. Experimental results on two datasets show that VolumeFormer outperforms existing patch-based and patch-free methods with a comparatively fast inference speed and relatively compact design. Code is available at: https://github.com/Dootmaan/VolumeFormer.

Anatomy-Guided Deep Learning Model for Accurate and Robust Gross Tumor Volume Segmentation in Lung Cancer Radiation Therapy

In lung cancer radiation therapy, clinicians must outline the gross tumor volume (GTV) precisely on the planning computed tomography (pCT) for accurate radiation dose delivery. However, due to the limited contrast between tumor and normal tissues in lung parenchyma, accurate delineation of tumor boundaries is difficult leading to large inter-observer variation. In this study, we develop an anatomy-guided lung GTV deep segmentation model using a training cohort of multi-center datasets. The quantitative segmentation performance is evaluated on an independent dataset, where the inter-observer delineation variation is also assessed. We collected and curated four publicly available lung datasets with GTV annotations (Lung-PET-CT-Dx, LIDC-IDRI, NSCLC-Radiogenomics and RIDER-CT) for deep learning model development. A total of 871 CT scans of patients, who were diagnosed with T1-T4 NSCLC, were available for training after data curation. The GTV annotations of primary tumor were examined and edited by two experienced radiation oncologists following the RTOG 1106 protocol. An anatomy-guided deep learning model was proposed, which consisted two deep networks. The first deep network used CT scan as input and segmented 4 anatomic organs (airway, heart, pulmonary artery and pulmonary vein), while the second deep network took both CT scan and these pre-segmented 4 organs as input and segmented the lung GTV. With the help of anatomic priors from 4 pre-segmented organs, the second deep network could more easily locate the GTV. We used nnUNet as the deep segmentation network. For evaluation, we used NSCLC-Radiomics as the testing dataset, which contains 20 CT scans each annotated by 5 radiation oncologists. The auto-segmented GTV were compared against each of the manual GTV reference. Inter-observer variation was also assessed using the 5 manual GTV references. The proposed anatomic-guided lung GTV segmentation model achieved a mean Dice score of 82.4% and 95% Hausdorff distance (HD95) of 6.9mm when averaged cross 20 patients and 5 GTV references (Table 1), which outperformed the basic deep GTV segmentation model by markedly reducing 19.4% HD95 error. The performance of proposed model was also comparable to the inter-observer variation (Dice score: 82.4% vs. 81.9%, HD95 6.9 vs. 6.4mm), indicating that our model had similar reproducibility as human observers. We developed and tested an anatomy-guided deep learning model for segmenting GTV in NSCLC patients. The model achieves high quantitative segmentation performance, which is comparable to the human observer variation. It can be potentially used in radiotherapy practice to improve GTV delineation consistency and reduce workloads of radiation oncologists.

Supervised Transfer Learning for Multi Organs 3D Segmentation With Registration Tools for Metal Artifact Reduction in CT Images

Radiological diagnostics are undeniably crucial in everyday healthcare. CT scans can sometimes provide misleading clues and delay successive treatment due to artifacts caused by reflected radiation from metallic implants. This work successfully segments multiple organs containing metal implants and discards artifacts using a combination of non-rigid transformations, Scribbles-based segmentation, and a pre-trained auto segmentation model (DynaUnet -Pretrained-Model). The presented transfer learning model combined the benefits of an interactive environment and reduced computational and processing-time costs. The transfer learning model proved high auto segmentation performance for multi-organs with metal implants' presence by decreasing metal artefact's impact on the segmentation process and the achieved segmentation accuracies between 0.9998 for the spleen and 0.9829 for the stomach.

Differential evolution-based neural architecture search for brain vessel segmentation

Brain vasculature analysis is critical in developing novel treatment targets for neurodegenerative diseases. Such an accurate analysis cannot be performed manually but requires a semi-automated or fully-automated approach. Deep learning methods have recently proven indispensable for the automated segmentation and analysis of medical images. However, optimizing a deep learning network architecture is another challenge. Manually selecting deep learning network architectures and tuning their hyper-parameters requires a lot of expertise and effort. To solve this problem, neural architecture search (NAS) approaches that explore more efficient network architectures with high segmentation performance have been proposed in the literature. This study introduces differential evolution-based NAS approaches in which a novel search space is proposed for brain vessel segmentation. We select two architectures that are frequently used for medical image segmentation, i.e. U-Net and Attention U-Net, as baselines for NAS optimizations. The conventional differential evolution and the opposition-based differential evolution with novel search space are employed as search methods in NAS. Furthermore, we perform ablation studies and evaluate the effects of specific loss functions, model pruning, threshold selection and generalization performance on the proposed models. The experiments are conducted on two datasets providing 335 single-channel 8-bit gray-scale images. These datasets are a public volumetric cerebrovascular system dataset (vesseINN) and our own dataset called KUVESG. The proposed NAS approaches, namely UNAS-Net and Attention UNAS-Net architectures, yield better segmentation performance in terms of different segmentation metrics. More specifically, UNAS-Net with differential evolution reveals high dice score/sensitivity values of 79.57/81.48, respectively. Moreover, they provide shorter inference times by a factor of 9.15 than the baseline methods.

From one field to another—Unsupervised domain adaptation for semantic segmentation in agricultural robotics

In traditional arable crop fields, tractors treat the whole field uniformly applying large quantities of herbicides and pesticides for weed control and plant protection. Autonomous robots, instead, offer the potential to provide a per-plant treatment, thus turning weed control and plant protection environment-friendly. To this end, an autonomous robot has to reliably distinguish crops, weeds, and soil under a diverse range of environmental conditions using its onboard sensors. Such recognition ability forms the basis for targeted plant-specific treatments in the form of spot applications. Basically, all such perception systems used today rely on some form of machine learning technique. However, current learning-based solutions often show a performance decay when applied under new field conditions. This is a major bottleneck for real-world application and finally commercial adoption. In this paper, we propose a simple yet effective approach to unsupervised domain adaptation for semantic segmentation systems so that an existing segmentation pipeline can be adapted to different fields, different robots, and different crops. Our system yields a high segmentation performance in new target fields without the need for extra manual annotations. It exploits only annotations from the source domain, i.e., the original field used for training the robot’s vision system. Our thorough evaluation shows that our approach achieves high accuracy when transferring an existing segmentation system to different environmental conditions, different plant species, and different robotic systems.

Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation.

Supervised deep learning-based methods yield accurate results for medical image segmentation. However, they require large labeled datasets for this, and obtaining them is a laborious task that requires clinical expertise. Semi/self-supervised learning-based approaches address this limitation by exploiting unlabeled data along with limited annotated data. Recent self-supervised learning methods use contrastive loss to learn good global level representations from unlabeled images and achieve high performance in classification tasks on popular natural image datasets like ImageNet. In pixel-level prediction tasks such as segmentation, it is crucial to also learn good local level representations along with global representations to achieve better accuracy. However, the impact of the existing local contrastive loss-based methods remains limited for learning good local representations because similar and dissimilar local regions are defined based on random augmentations and spatial proximity; not based on the semantic label of local regions due to lack of large-scale expert annotations in the semi/self-supervised setting. In this paper, we propose a local contrastive loss to learn good pixel level features useful for segmentation by exploiting semantic label information obtained from pseudo-labels of unlabeled images alongside limited annotated images with ground truth (GT) labels. In particular, we define the proposed contrastive loss to encourage similar representations for the pixels that have the same pseudo-label/GT label while being dissimilar to the representation of pixels with different pseudo-label/GT label in the dataset. We perform pseudo-label based self-training and train the network by jointly optimizing the proposed contrastive loss on both labeled and unlabeled sets and segmentation loss on only the limited labeled set. We evaluated the proposed approach on three public medical datasets of cardiac and prostate anatomies, and obtain high segmentation performance with a limited labeled set of one or two 3D volumes. Extensive comparisons with the state-of-the-art semi-supervised and data augmentation methods and concurrent contrastive learning methods demonstrate the substantial improvement achieved by the proposed method. The code is made publicly available at https://github.com/krishnabits001/pseudo_label_contrastive_training.

A Lightweight Convolutional Neural Network Based on Dynamic Level-Set Loss Function for Spine MR Image Segmentation.

Spine MR image segmentation is important foundation for computer-aided diagnostic (CAD) algorithms of spine disorders. Convolutional neural networks segment effectively, but require high computational costs. To design a lightweight model based on dynamic level-set loss function for high segmentation performance. Retrospective. Four hundred forty-eight subjects (3163 images) from two separate datasets. Dataset-1: 276 subjects/994 images (53.26% female, mean age 49.02 ± 14.09), all for disc degeneration screening, 188 had disc degeneration, 67 had herniated disc. Dataset-2: public dataset with 172 subjects/2169 images, 142 patients with vertebral degeneration, 163 patients with disc degeneration. T2 weighted turbo spin echo sequences at 3T. Dynamic Level-set Net (DLS-Net) was compared with four mainstream (including U-net++) and four lightweight models, and manual label made by five radiologists (vertebrae, discs, spinal fluid) used as segmentation evaluation standard. Five-fold cross-validation are used for all experiments. Based on segmentation, a CAD algorithm of lumbar disc was designed for assessing DLS-Net's practicality, and the text annotation (normal, bulging, or herniated) from medical history data were used as evaluation standard. All segmentation models were evaluated with DSC, accuracy, precision, and AUC. The pixel numbers of segmented results were compared with manual label using paired t-tests, with P < 0.05 indicating significance. The CAD algorithm was evaluated with accuracy of lumbar disc diagnosis. With only 1.48% parameters of U-net++, DLS-Net achieved similar accuracy in both datasets (Dataset-1: DSC 0.88 vs. 0.89, AUC 0.94 vs. 0.94; Dataset-2: DSC 0.86 vs. 0.86, AUC 0.93 vs. 0.93). The segmentation results of DLS-Net showed no significant differences with manual labels in pixel numbers for discs (Dataset-1: 1603.30 vs. 1588.77, P = 0.22; Dataset-2: 863.61 vs. 886.4, P = 0.14) and vertebrae (Dataset-1: 3984.28 vs. 3961.94, P = 0.38; Dataset-2: 4806.91 vs. 4732.85, P = 0.21). Based on DLS-Net's segmentation results, the CAD algorithm achieved higher accuracy than using non-cropped MR images (87.47% vs. 61.82%). The proposed DLS-Net has fewer parameters but achieves similar accuracy to U-net++, helps CAD algorithm achieve higher accuracy, which facilitates wider application. 2 TECHNICAL EFFICACY: Stage 1.

Scanner agnostic large-scale evaluation of MS lesion delineation tool for clinical MRI.

Patients with MS are MRI scanned continuously throughout their disease course resulting in a large manual workload for radiologists which includes lesion detection and size estimation. Though many models for automatic lesion segmentation have been published, few are used broadly in clinic today, as there is a lack of testing on clinical datasets. By collecting a large, heterogeneous training dataset directly from our MS clinic we aim to present a model which is robust to different scanner protocols and artefacts and which only uses MRI modalities present in routine clinical examinations. We retrospectively included 746 patients from routine examinations at our MS clinic. The inclusion criteria included acquisition at one of seven different scanners and an MRI protocol including 2D or 3D T2-w FLAIR, T2-w and T1-w images. Reference lesion masks on the training (n = 571) and validation (n = 70) datasets were generated using a preliminary segmentation model and subsequent manual correction. The test dataset (n = 100) was manually delineated. Our segmentation model https://github.com/CAAI/AIMS/ was based on the popular nnU-Net, which has won several biomedical segmentation challenges. We tested our model against the published segmentation models HD-MS-Lesions, which is also based on nnU-Net, trained with a more homogenous patient cohort. We furthermore tested model robustness to data from unseen scanners by performing a leave-one-scanner-out experiment. We found that our model was able to segment MS white matter lesions with a performance comparable to literature: DSC = 0.68, precision = 0.90, recall = 0.70, f1 = 0.78. Furthermore, the model outperformed HD-MS-Lesions in all metrics except precision = 0.96. In the leave-one-scanner-out experiment there was no significant change in performance (p < 0.05) between any of the models which were only trained on part of the dataset and the full segmentation model. In conclusion we have seen, that by including a large, heterogeneous dataset emulating clinical reality, we have trained a segmentation model which maintains a high segmentation performance while being robust to data from unseen scanners. This broadens the applicability of the model in clinic and paves the way for clinical implementation.

TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency

Accurate segmentation of brain tumors from medical images is important for diagnosis and treatment planning, and it often requires multi-modal or contrast-enhanced images. However, in practice some modalities of a patient may be absent. Synthesizing the missing modality has a potential for filling this gap and achieving high segmentation performance. Existing methods often treat the synthesis and segmentation tasks separately or consider them jointly but without effective regularization of the complex joint model, leading to limited performance. We propose a novel brain Tumor Image Synthesis and Segmentation network (TISS-Net) that obtains the synthesized target modality and segmentation of brain tumors end-to-end with high performance. First, we propose a dual-task-regularized generator that simultaneously obtains a synthesized target modality and a coarse segmentation, which leverages a tumor-aware synthesis loss with perceptibility regularization to minimize the high-level semantic domain gap between synthesized and real target modalities. Based on the synthesized image and the coarse segmentation, we further propose a dual-task segmentor that predicts a refined segmentation and error in the coarse segmentation simultaneously, where a consistency between these two predictions is introduced for regularization. Our TISS-Net was validated with two applications: synthesizing FLAIR images for whole glioma segmentation, and synthesizing contrast-enhanced T1 images for Vestibular Schwannoma segmentation. Experimental results showed that our TISS-Net largely improved the segmentation accuracy compared with direct segmentation from the available modalities, and it outperformed state-of-the-art image synthesis-based segmentation methods.