Deep Learning-based Segmentation Research Articles

Indoor Localization of Hand-Held OCT Probe Using Visual Odometry and Real-Time Segmentation Using Deep Learning.

Optical coherence tomography (OCT) is an established medical imaging modality that has found widespread use due to its ability to visualize tissue structures at a high resolution. Currently, OCT hand-held imaging probes lack positional information, making it difficult or even impossible to link a specific image to the location it was originally obtained. In this study, we propose a camera-based localization method to track and record the scanner position in real-time, as well as providing a deep learning-based segmentation method. We used camera-based visual odometry (VO) and simultaneous mapping and localization (SLAM) to compute and visualize the location of a hand-held OCT imaging probe. A deep convolutional neural network (CNN) was used for kidney tubule lumens segmentation. The mean absolute error (MAE) and the standard deviation (STD) for 1D translation were found to be 0.15 mm and 0.26mm respectively. For 2D translation, the MAE and STD were found to be 0.85 mm and 0.50 mm, respectively. The dice coefficient of the segmentation method was 0.7. The t-statistic of the T-test between predicted and actual average densities and predicted and actual average diameters were 7.7547e-13 and 2.2288e-15 respectively. We also experimented on a preserved kidney utilizing our localization method with automatic segmentation. Comparisons of the average density maps and average diameter maps were made between the 3D comprehensive scan and VO system scan. Our results demonstrate that VO can track the probe location at high accuracy, and provides a user-friendly visualization tool to review OCT 2D images in 3D space. It also indicates that deep learning can provide high accuracy and high speed for segmentation. The proposed methods can be potentially used to predict delayed graft function (DGF) in kidney transplantation.

SuRVoS 2: Accelerating Annotation and Segmentation for Large Volumetric Bioimage Workflows Across Modalities and Scales.

As sample preparation and imaging techniques have expanded and improved to include a variety of options for larger sized and numbers of samples, the bottleneck in volumetric imaging is now data analysis. Annotation and segmentation are both common, yet difficult, data analysis tasks which are required to bring meaning to the volumetric data. The SuRVoS application has been updated and redesigned to provide access to both manual and machine learning-based segmentation and annotation techniques, including support for crowd sourced data. Combining adjacent, similar voxels (supervoxels) provides a mechanism for speeding up segmentation both in the painting of annotation and by training a segmentation model on a small amount of annotation. The support for layers allows multiple datasets to be viewed and annotated together which, for example, enables the use of correlative data (e.g. crowd-sourced annotations or secondary imaging techniques) to guide segmentation. The ability to work with larger data on high-performance servers with GPUs has been added through a client-server architecture and the Pytorch-based image processing and segmentation server is flexible and extensible, and allows the implementation of deep learning-based segmentation modules. The client side has been built around Napari allowing integration of SuRVoS into an ecosystem for open-source image analysis while the server side has been built with cloud computing and extensibility through plugins in mind. Together these improvements to SuRVoS provide a platform for accelerating the annotation and segmentation of volumetric and correlative imaging data across modalities and scales.

Cross-sectional and longitudinal Biomarker extraction and analysis for multicentre FLAIR brain MRI

Fluid-attenuated inversion recovery (FLAIR) MRI has emerged as an important sequence for the analysis of cerebrovascular (CVD) and Alzheimer's disease (AD). Large-scale, automated cross-sectional and longitudinal cerebral biomarker extraction from FLAIR datasets could progress disease characterization, improve disease monitoring, and help to determine optimal intervention times. Despite this, most automated biomarker extraction algorithms are designed for T1-weighted or multi-modal inputs. In this work, automated tools were used to extract biomarkers from large, FLAIR-only datasets to evaluate the feasibility of this sequence to characterize healthy, AD, and CVD subjects in a similar manner to traditional approaches. Total brain volume (TBV), cerebrospinal fluid (CSF) volume, and white matter lesion (WML) volume was measured for the cross-sectional biomarkers and the corresponding annual rates of change over multiple scans represented the longitudinal biomarkers. Biomarkers were extracted from two dementia cohorts (4356 vol, 162 233 images) and one vascular disease cohort (869 vol, 42 850 images) using deep learning-based segmentation algorithms designed specifically for FLAIR. Biomarkers from all cohorts were summarized using descriptive statistics, correlation analysis, and ANCOVA to assess differences in diagnostic labels while accounting for demographic and acquisition factors. Biomarkers from FLAIR MRI had similar trends with those extracted from traditional modalities in the literature for characterizing healthy, AD, and CVD subjects. This demonstrates that FLAIR MRI can be used for end-to-end analysis of large AD and CVD datasets, which can lower acquisition costs, simplify clinical translation, and reduce measurement error associated with multi-modal approaches.

An Improved 3D Deep Learning-Based Segmentation of Left Ventricular Myocardial Diseases from Delayed-Enhancement MRI with Inclusion and Classification Prior Information U-Net (ICPIU-Net).

Accurate segmentation of the myocardial scar may supply relevant advancements in predicting and controlling deadly ventricular arrhythmias in subjects with cardiovascular disease. In this paper, we propose the architecture of inclusion and classification of prior information U-Net (ICPIU-Net) to efficiently segment the left ventricle (LV) myocardium, myocardial infarction (MI), and microvascular-obstructed (MVO) tissues from late gadolinium enhancement magnetic resonance (LGE-MR) images. Our approach was developed using two subnets cascaded to first segment the LV cavity and myocardium. Then, we used inclusion and classification constraint networks to improve the resulting segmentation of the diseased regions within the pre-segmented LV myocardium. This network incorporates the inclusion and classification information of the LGE-MRI to maintain topological constraints of pathological areas. In the testing stage, the outputs of each segmentation network obtained with specific estimated parameters from training were fused using the majority voting technique for the final label prediction of each voxel in the LGE-MR image. The proposed method was validated by comparing its results to manual drawings by experts from 50 LGE-MR images. Importantly, compared to various deep learning-based methods participating in the EMIDEC challenge, the results of our approach have a more significant agreement with manual contouring in segmenting myocardial diseases.

Semantic segmentation of multispectral photoacoustic images using deep learning

Photoacoustic (PA) imaging has the potential to revolutionize functional medical imaging in healthcare due to the valuable information on tissue physiology contained in multispectral photoacoustic measurements. Clinical translation of the technology requires conversion of the high-dimensional acquired data into clinically relevant and interpretable information. In this work, we present a deep learning-based approach to semantic segmentation of multispectral photoacoustic images to facilitate image interpretability. Manually annotated photoacoustic and ultrasound imaging data are used as reference and enable the training of a deep learning-based segmentation algorithm in a supervised manner. Based on a validation study with experimentally acquired data from 16 healthy human volunteers, we show that automatic tissue segmentation can be used to create powerful analyses and visualizations of multispectral photoacoustic images. Due to the intuitive representation of high-dimensional information, such a preprocessing algorithm could be a valuable means to facilitate the clinical translation of photoacoustic imaging.

Deep learning-based simultaneous registration and unsupervised non-correspondence segmentation of medical images with pathologies

PurposeThe registration of medical images often suffers from missing correspondences due to inter-patient variations, pathologies and their progression leading to implausible deformations that cause misregistrations and might eliminate valuable information. Detecting non-corresponding regions simultaneously with the registration process helps generating better deformations and has been investigated thoroughly with classical iterative frameworks but rarely with deep learning-based methods.MethodsWe present the joint non-correspondence segmentation and image registration network (NCR-Net), a convolutional neural network (CNN) trained on a Mumford–Shah-like functional, transferring the classical approach to the field of deep learning. NCR-Net consists of one encoding and two decoding parts allowing the network to simultaneously generate diffeomorphic deformations and segment non-correspondences. The loss function is composed of a masked image distance measure and regularization of deformation field and segmentation output. Additionally, anatomical labels are used for weak supervision of the registration task. No manual segmentations of non-correspondences are required.ResultsThe proposed network is evaluated on the publicly available LPBA40 dataset with artificially added stroke lesions and a longitudinal optical coherence tomography (OCT) dataset of patients with age-related macular degeneration. The LPBA40 data are used to quantitatively assess the segmentation performance of the network, and it is shown qualitatively that NCR-Net can be used for the unsupervised segmentation of pathologies in OCT images. Furthermore, NCR-Net is compared to a registration-only network and state-of-the-art registration algorithms showing that NCR-Net achieves competitive performance and superior robustness to non-correspondences.ConclusionNCR-Net, a CNN for simultaneous image registration and unsupervised non-correspondence segmentation, is presented. Experimental results show the network’s ability to segment non-correspondence regions in an unsupervised manner and its robust registration performance even in the presence of large pathologies.

Learning a Discriminative Feature Attention Network for pancreas CT segmentation

Accurate pancreas segmentation is critical for the diagnosis and management of diseases of the pancreas. It is challenging to precisely delineate pancreas due to the highly variations in volume, shape and location. In recent years, coarse-to-fine methods have been widely used to alleviate class imbalance issue and improve pancreas segmentation accuracy. However, cascaded methods could be computationally intensive and the refined results are significantly dependent on the performance of its coarse segmentation results. To balance the segmentation accuracy and computational efficiency, we propose a Discriminative Feature Attention Network for pancreas segmentation, to effectively highlight pancreas features and improve segmentation accuracy without explicit pancreas location. The final segmentation is obtained by applying a simple yet effective post-processing step. Two experiments on both public NIH pancreas CT dataset and abdominal BTCV multi-organ dataset are individually conducted to show the effectiveness of our method for 2D pancreas segmentation. We obtained average Dice Similarity Coefficient (DSC) of 82.82±6.09%, average Jaccard Index (JI) of 71.13± 8.30% and average Symmetric Average Surface Distance (ASD) of 1.69 ± 0.83 mm on the NIH dataset. Compared to the existing deep learning-based pancreas segmentation methods, our experimental results achieve the best average DSC and JI value.

Deep learning-based segmentation of multiple species of weeds and corn crop using synthetic and real image datasets

• Identification and segmentation of weed species is of great importance. • Semantic segmentation on natural images requires enormous manual annotation. • The proposed architecture increases the performance in distinguishing between weed species. • The proposed methodology reduces the annotation effort to one third. • The use of synthetic datasets doubles the segmentation performance. Weeds compete with productive crops for soil, nutrients and sunlight and are therefore a major contributor to crop yield loss, which is why safer and more effective herbicide products are continually being developed. Digital evaluation tools to automate and homogenize field measurements are of vital importance to accelerate their development. However, the development of these tools requires the generation of semantic segmentation datasets, which is a complex, time-consuming and not easily affordable task. In this paper, we present a deep learning segmentation model that is able to distinguish between different plant species at the pixel level. First, we have generated three extensive datasets targeting one crop species ( Zea mays ), three grass species ( Setaria verticillata, Digitaria sanguinalis, Echinochloa crus-galli ) and three broadleaf species ( Abutilon theophrasti, Chenopodium albums, Amaranthus retroflexus ). The first dataset consists of real field images that were manually annotated. The second dataset is composed of images of plots where only one species is present at a time and the third type of dataset was synthetically generated from images of individual plants mimicking the distribution of real field images. Second, we have proposed a semantic segmentation architecture by extending a PSPNet architecture with an auxiliary classification loss to aid model convergence. Our results show that the network performance increases when supplementing the real field image dataset with the other types of datasets without increasing the manual annotation effort. More specifically, the use of the real field dataset obtains a Dice-Søensen Coefficient (DSC) score of 25.32 . This performance increases when this dataset is combined with the single-species class dataset (DSC= 47.97 ) or the synthetic dataset (DSC= 45.20 ). As for the proposed model, the ablation method shows that by removing the proposed auxiliary classification loss, the segmentation performance decreases (DSC= 45.96 ) compared to the proposed architecture method (DSC= 47.97 ). The proposed method shows better performance than the current state of the art. In addition, the use of proposed single-species or synthetic datasets can double the performance of the algorithm than when using real datasets without additional manual annotation effort.

Deep Learning-Based Segmentation and Classification of COVID-19 Infection Severity Levels from CT Scans

Due to the emergence of coronavirus disease 2019 (COVID-19) around the globe, the detection and treatment of COVID-19 patients is an extremely essential process in healthcare systems. Among many imaging technologies, Computed Tomography (CT) analyses with deep learning frameworks offer better efficiency than the other imaging modalities. To segment the infected COVID-19 Region-Of-Interests (ROIs), an Enriched 2.5D U-Net (E2.5D U-Net)-based deep learner has been used. In contrast, it requires categorization of infection severity to identify the patients in the prior stage. Hence, in this article, feature extraction with a classification framework is proposed to learn deep features related to the infection disease severity. In this framework, each segment from the E2.5D U-Net is fed to a deep learner such as DenseNet201 to extract the deep features. These features are learned independently by different machine learning classifiers to categorize the infection severity levels. It aids physicians in diagnosing COVID-19 patients in advance. But it needs other infection-related features to enhance the efficiency. Therefore, a multi-modeling classification framework is proposed. This framework extracts the handcrafted features from CT scans and concatenates them with the deep features to get the unified feature vector. Moreover, these feature vectors are trained by using the multi-modeling classifier for predicting the infection severity levels with higher accuracy. At last, the testing outcomes exhibit that the multi-modal classifier establishes a higher efficiency than the standard classifier frameworks.

One-Shot Learning with Pseudo-Labeling for Cattle Video Segmentation in Smart Livestock Farming.

Simple SummaryDeep learning-based segmentation methods rely on large-scale pixel-labeled datasets to achieve good performance. However, it is resource-costly to label animal images due to their irregular contours and changing postures. To keep a balance between segmentation accuracy and speed using limited label data, we propose a one-shot learning-based approach with pseudo-labeling to segment animals in videos, relying on only one labeled frame. Experiments were conducted on a challenging feedlot cattle video dataset acquired by the authors, and the results show that the proposed method outperformed state-of-the-art methods such as one-shot video object segmentation (OSVOS) and one-shot modulation network (OSMN). Our proposed one-shot learning with pseudo-labeling reduces the reliance on labeled data and could serve as an enabling component for smart farming-related applications.Computer vision-based technologies play a key role in precision livestock farming, and video-based analysis approaches have been advocated as useful tools for automatic animal monitoring, behavior analysis, and efficient welfare measurement management. Accurately and efficiently segmenting animals’ contours from their backgrounds is a prerequisite for vision-based technologies. Deep learning-based segmentation methods have shown good performance through training models on a large amount of pixel-labeled images. However, it is challenging and time-consuming to label animal images due to their irregular contours and changing postures. In order to reduce the reliance on the number of labeled images, one-shot learning with a pseudo-labeling approach is proposed using only one labeled image frame to segment animals in videos. The proposed approach is mainly comprised of an Xception-based Fully Convolutional Neural Network (Xception-FCN) module and a pseudo-labeling (PL) module. Xception-FCN utilizes depth-wise separable convolutions to learn different-level visual features and localize dense prediction based on the one single labeled frame. Then, PL leverages the segmentation results of the Xception-FCN model to fine-tune the model, leading to performance boosts in cattle video segmentation. Systematic experiments were conducted on a challenging feedlot cattle video dataset acquired by the authors, and the proposed approach achieved a mean intersection-over-union score of 88.7% and a contour accuracy of 80.8%, outperforming state-of-the-art methods (OSVOS and OSMN). Our proposed one-shot learning approach could serve as an enabling component for livestock farming-related segmentation and detection applications.

A Quantitative Imaging Biomarker Supporting Radiological Assessment of Hippocampal Sclerosis Derived From Deep Learning-Based Segmentation of T1w-MRI.

PurposeHippocampal volumetry is an important biomarker to quantify atrophy in patients with mesial temporal lobe epilepsy. We investigate the sensitivity of automated segmentation methods to support radiological assessments of hippocampal sclerosis (HS). Results from FreeSurfer and FSL-FIRST are contrasted to a deep learning (DL)-based segmentation method.Materials and MethodsWe used T1-weighted MRI scans from 105 patients with epilepsy and 354 healthy controls. FreeSurfer, FSL, and a DL-based method were applied for brain anatomy segmentation. We calculated effect sizes (Cohen's d) between left/right HS and healthy controls based on the asymmetry of hippocampal volumes. Additionally, we derived 14 shape features from the segmentations and determined the most discriminating feature to identify patients with hippocampal sclerosis by a support vector machine (SVM).ResultsDeep learning-based segmentation of the hippocampus was the most sensitive to detecting HS. The effect sizes of the volume asymmetries were larger with the DL-based segmentations (HS left d= −4.2, right = 4.2) than with FreeSurfer (left= −3.1, right = 3.7) and FSL (left= −2.3, right = 2.5). For the classification based on the shape features, the surface-to-volume ratio was identified as the most important feature. Its absolute asymmetry yielded a higher area under the curve (AUC) for the deep learning-based segmentation (AUC = 0.87) than for FreeSurfer (0.85) and FSL (0.78) to dichotomize HS from other epilepsy cases. The robustness estimated from repeated scans was statistically significantly higher with DL than all other methods.ConclusionOur findings suggest that deep learning-based segmentation methods yield a higher sensitivity to quantify hippocampal sclerosis than atlas-based methods and derived shape features are more robust. We propose an increased asymmetry in the surface-to-volume ratio of the hippocampus as an easy-to-interpret quantitative imaging biomarker for HS.

Simple Code Implementation for Deep Learning-Based Segmentation to Evaluate Central Serous Chorioretinopathy in Fundus Photography.

PurposeCentral serous chorioretinopathy (CSC) is a retinal disease that frequently shows resolution and recurrence with serous detachment of the neurosensory retina. Here, we present a deep learning analysis of subretinal fluid (SRF) lesion segmentation in fundus photographs to evaluate CSC.MethodsWe collected 194 fundus photographs of SRF lesions from the patients with CSC. Three graders manually annotated of the entire SRF area in the retinal images. The dataset was randomly separated into training (90%) and validation (10%) datasets. We used the U-Net segmentation model based on conditional generative adversarial networks (pix2pix) to detect the SRF lesions. The algorithms were trained and validated using Google Colaboratory. Researchers did not need prior knowledge of coding skills or computing resources to implement this code.ResultsThe validation results showed that the Jaccard index and Dice coefficient scores were 0.619 and 0.763, respectively. In most cases, the segmentation results overlapped with most of the reference areas in the annotated images. However, cases with exceptional SRFs were not accurate in terms of prediction. Using Colaboratory, the proposed segmentation task ran easily in a web-based environment without setup or personal computing resources.ConclusionsThe results suggest that the deep learning model based on U-Net from the pix2pix algorithm is suitable for the automatic segmentation of SRF lesions to evaluate CSC.Translational RelevanceOur code implementation has the potential to facilitate ophthalmology research; in particular, deep learning–based segmentation can assist in the development of pathological lesion detection solutions.

FastSurferVINN: Building resolution-independence into deep learning segmentation methods—A solution for HighRes brain MRI

Leading neuroimaging studies have pushed 3T MRI acquisition resolutions below 1.0 mm for improved structure definition and morphometry. Yet, only few, time-intensive automated image analysis pipelines have been validated for high-resolution (HiRes) settings. Efficient deep learning approaches, on the other hand, rarely support more than one fixed resolution (usually 1.0 mm). Furthermore, the lack of a standard submillimeter resolution as well as limited availability of diverse HiRes data with sufficient coverage of scanner, age, diseases, or genetic variance poses additional, unsolved challenges for training HiRes networks. Incorporating resolution-independence into deep learning-based segmentation, i.e., the ability to segment images at their native resolution across a range of different voxel sizes, promises to overcome these challenges, yet no such approach currently exists. We now fill this gap by introducing a Voxel-size Independent Neural Network (VINN) for resolution-independent segmentation tasks and present FastSurferVINN, which (i) establishes and implements resolution-independence for deep learning as the first method simultaneously supporting 0.7–1.0 mm whole brain segmentation, (ii) significantly outperforms state-of-the-art methods across resolutions, and (iii) mitigates the data imbalance problem present in HiRes datasets. Overall, internal resolution-independence mutually benefits both HiRes and 1.0 mm MRI segmentation. With our rigorously validated FastSurferVINN we distribute a rapid tool for morphometric neuroimage analysis. The VINN architecture, furthermore, represents an efficient resolution-independent segmentation method for wider application.

Deep learning-based segmentation of the thorax in mouse micro-CT scans

For image-guided small animal irradiations, the whole workflow of imaging, organ contouring, irradiation planning, and delivery is typically performed in a single session requiring continuous administration of anaesthetic agents. Automating contouring leads to a faster workflow, which limits exposure to anaesthesia and thereby, reducing its impact on experimental results and on animal wellbeing. Here, we trained the 2D and 3D U-Net architectures of no-new-Net (nnU-Net) for autocontouring of the thorax in mouse micro-CT images. We trained the models only on native CTs and evaluated their performance using an independent testing dataset (i.e., native CTs not included in the training and validation). Unlike previous studies, we also tested the model performance on an external dataset (i.e., contrast-enhanced CTs) to see how well they predict on CTs completely different from what they were trained on. We also assessed the interobserver variability using the generalized conformity index (hbox {CI}_{mathrm{gen}}) among three observers, providing a stronger human baseline for evaluating automated contours than previous studies. Lastly, we showed the benefit on the contouring time compared to manual contouring. The results show that 3D models of nnU-Net achieve superior segmentation accuracy and are more robust to unseen data than 2D models. For all target organs, the mean surface distance (MSD) and the Hausdorff distance (95p HD) of the best performing model for this task (nnU-Net 3d_fullres) are within 0.16 mm and 0.60 mm, respectively. These values are below the minimum required contouring accuracy of 1 mm for small animal irradiations, and improve significantly upon state-of-the-art 2D U-Net-based AIMOS method. Moreover, the conformity indices of the 3d_fullres model also compare favourably to the interobserver variability for all target organs, whereas the 2D models perform poorly in this regard. Importantly, the 3d_fullres model offers 98% reduction in contouring time.

Fully automated radiomic screening pipeline for osteoporosis and abnormal bone density with a deep learning-based segmentation using a short lumbar mDixon sequence.

Although lumbar bone marrow fat fraction (BMFF) has been demonstrated to be predictive of osteoporosis, its utility is limited by the requirement of manual segmentation. Additionally, quantitative features beyond simple BMFF average remain to be explored. In this study, we developed a fully automated radiomic pipeline using deep learning-based segmentation to detect osteoporosis and abnormal bone density (ABD) using a <20 s modified Dixon (mDixon) sequence. In total, 222 subjects underwent quantitative computed tomography (QCT) and lower back magnetic resonance imaging (MRI). Bone mineral density (BMD) were extracted from L1-L3 using QCT as the reference standard; 206 subjects (48.8±14.9 years old, 140 females) were included in the final analysis, and were divided temporally into the training/validation set (142/64 subjects). A deep-learning network was developed to perform automated segmentation. Radiomic models were built using the same training set to predict ABD and osteoporosis using the mDixon maps. The performance was evaluated using the temporal validation set comprised of 64 subjects, along with the automated segmentation. Additional 25 subjects (56.1±8.8 years, 14 females) from another site and a different scanner vendor was included as independent validation to evaluate the performance of the pipeline. The automated segmentation achieved an outstanding mean dice coefficient of 0.912±0.062 compared to manual in the temporal validation. Task-based evaluation was performed in the temporal validation set, for predicting ABD and osteoporosis, the area under the curve, sensitivity, specificity, and accuracy were 0.925/0.899, 0.923/0.667, 0.789/0.873, 0.844/0.844, respectively. These values were comparable to that of manual segmentation. External validation (cross-vendor) was also performed; the area under the curve, sensitivity, specificity, and accuracy were 0.688/0.913, 0.786/0.857, 0.545/0.944, 0.680/0.920 for ABD and osteoporosis prediction, respectively. Our work is the first attempt using radiomics to predict osteoporosis with BMFF map, and the deep-learning based segmentation will further facilitate the clinical utility of the pipeline as a screening tool for early detection of ABD.

Microstructure evolution of lithium-ion battery electrodes at different states of charge: Deep learning-based segmentation

The evolution of the microstructure of a battery electrode is closely related to battery performance. Characterization and visualization of the evolution of the microstructure is essential for optimization of manufactured electrodes. The validity of the battery structure representation affects the accuracy of the extracted microstructure parameters. In this study, a mini-cylindrical battery is designed to allow microstructure parameters to be obtained at different states of charge, bearing in mind the influence of the real battery structure. An argon-ion cross-section polisher is used to obtain a large area of the electrode for observation. In addition, an image segmentation method based on a modified U-Net neural network is developed to enhance the quality of the extracted microstructure. The relationship between porosity and thickness at different states of electrode charge is presented through experiments and deep learning of images. This method provides new insight into the evolution of electrode microstructure and can potentially guide the manufacturing of lithium-ion batteries.

Structure attention co-training neural network for neovascularization segmentation in intravascular optical coherence tomography.

To development and validate a neovascularization (NV) segmentation model in intravascular optical coherence tomography (IVOCT) through deep learning methods. A total of 1950 2D slices of 70 IVOCT pullbacks were used in our study. We randomly selected 1273 2D slices from 44 patients as the training set, 379 2D slices from 11 patients as the validation set, and 298 2D slices from the last 15 patients as the testing set. Automatic NV segmentation is quite challenging, as it must address issues of speckle noise, shadow artifacts, high distribution variation, etc. To meet these challenges, a new deep learning-based segmentation method is developed based on a co-training architecture with an integrated structural attention mechanism. Co-training is developed to exploit the features of three consecutive slices. The structural attention mechanism comprises spatial and channel attention modules and is integrated into the co-training architecture at each up-sampling step. A cascaded fixed network is further incorporated to achieve segmentation at the image level in a coarse-to-fine manner. Extensive experiments were performed involving a comparison with several state-of-the-art deep learning-based segmentation methods. Moreover, the consistency of the results with those of manual segmentation was also investigated. Our proposed NV automatic segmentation method achieved the highest correlation with the manual delineation by interventional cardiologists (the Pearson correlation coefficient is 0.825). In this work, we proposed a co-training architecture with an integrated structural attention mechanism to segment NV in IVOCT images. The good agreement between our segmentation results and manual segmentation indicates that the proposed method has great potential for application in the clinical investigation of NV-related plaque diagnosis and treatment.

Code-Free Development and Deployment of Deep Segmentation Models for Digital Pathology.

Application of deep learning on histopathological whole slide images (WSIs) holds promise of improving diagnostic efficiency and reproducibility but is largely dependent on the ability to write computer code or purchase commercial solutions. We present a code-free pipeline utilizing free-to-use, open-source software (QuPath, DeepMIB, and FastPathology) for creating and deploying deep learning-based segmentation models for computational pathology. We demonstrate the pipeline on a use case of separating epithelium from stroma in colonic mucosa. A dataset of 251 annotated WSIs, comprising 140 hematoxylin-eosin (HE)-stained and 111 CD3 immunostained colon biopsy WSIs, were developed through active learning using the pipeline. On a hold-out test set of 36 HE and 21 CD3-stained WSIs a mean intersection over union score of 95.5 and 95.3% was achieved on epithelium segmentation. We demonstrate pathologist-level segmentation accuracy and clinical acceptable runtime performance and show that pathologists without programming experience can create near state-of-the-art segmentation solutions for histopathological WSIs using only free-to-use software. The study further demonstrates the strength of open-source solutions in its ability to create generalizable, open pipelines, of which trained models and predictions can seamlessly be exported in open formats and thereby used in external solutions. All scripts, trained models, a video tutorial, and the full dataset of 251 WSIs with ~31 k epithelium annotations are made openly available at https://github.com/andreped/NoCodeSeg to accelerate research in the field.

ThoraxNet: a 3D U-Net based two-stage framework for OAR segmentation on thoracic CT images.

An important phase of radiation treatment planning is the accurate contouring of the organs at risk (OAR), which is necessary for the dose distribution calculation. The manual contouring approach currently used in clinical practice is tedious, time-consuming, and prone to inter and intra-observer variation. Therefore, a deep learning-based auto contouring tool can solve these issues by accurately delineating OARs on the computed tomography (CT) images. This paper proposes a two-stage deep learning-based segmentation model with an attention mechanism that automatically delineates OARs in thoracic CT images. After preprocessing the input CT volume, a 3D U-Net architecture will locate each organ to generate cropped images for the segmentation network. Next, two differently configured U-Net-based networks will perform the segmentation of large organs-left lung, right lung, heart, and small organs-esophagus and spinal cord, respectively. A post-processing step integrates all the individually-segmented organs to generate the final result. The suggested model outperformed the state-of-the-art approaches in terms of dice similarity coefficient (DSC) values for the lungs and the heart. It is worth mentioning that the proposed model achieved a dice score of 0.941, which is 1.1% higher than the best previous dice score, in the case of the heart, an important organ in the human body. Moreover, the clinical acceptance of the results is verified using dosimetric analysis. To delineate all five organs on a CT scan of size [Formula: see text], our model takes only 8.61s. The proposed open-source automatic contouring tool can generate accurate contours in minimal time, consequently speeding up the treatment time and reducing the treatment cost.

Volumetric segmentation of ground glass nodule based on 3D attentional cascaded residual U‐Net and conditional random field

Ground glass nodule (GGN) segmentation is one of the important and challenging tasks in diagnosing early-stage lung adenocarcinomas. Manually delineating of 3D GGN in a computed tomography (CT) image is a subjective, laborious, and tedious task, which presents poor repeatability. To reduce the annotation burden and improve the segmentation performance, this study proposes a 3D deep learning-based volumetric segmentation model to segment the GGN in CT images. A total of 379 GGNs were retrospectively collected from the public database, Shanghai Pulmonary Hospital (SHPH), and Fudan University Shanghai Cancer Center (FUSCC). First, a series of image preprocessing techniques involving image resampling, intensity normalization, 3D nodule patch cropping, and data augmentation, were adopted to generate the input images for the deep learning model by using CT scans. Then, a 3D attentional cascaded residual network (ACRU-Net) was proposed to develop the deep learning-based segmentation model by using the residual network and the atrous spatial pyramid pooling module. To improve the model performance, a voxel-based conditional random field (CRF) method was used to optimize the segmentation results. Finally, a balanced cross-entropy and Dice combined loss function was applied to train and build the segmentation model. Testing on SHPH and FUSCC datasets, the proposed method generates the Dice coefficients of 0.721±0.167 and 0.733±0.100, respectively, which are higher than those of 3D residual U-Net and ACRU-Net without CRF optimization. The results demonstrated that combining 3D ACRU-Net and CRF effectively improved the GGN segmentation performance. The proposed segmentation model may provide a potential tool to help the radiologist in the segmentation and diagnosis of 3D GGN.