Average Dice Coefficient Research Articles

EANet: Multiscale autoencoder based edge attention network for fluid segmentation from SD‐OCT images

AbstractMacular edema is a retinal complication that occurs due to the presence of excess fluid between the retinal layers. This might lead to swelling in the retina and cause severe vision impairment if not detected in its early stages. This paper presents a robust Edge Attention network (EANet) for segmenting the different retinal fluids like Intraretinal Fluid (IRF), Subretinal Fluid (SRF), and Pigment Epithelial Detachment (PED) from the Spectral Domain – Optical Coherence Tomography (SD‐OCT) images. The proposed method employs a novel image enhancement technique by filtering OCT images using a BM3D (Block Matching and 3D Filtering) filter followed by Contrast Limited Adaptive Histogram Equalization (CLAHE) and a linear filter based on multivariate Taylor series to acquire the edge maps of the OCT images. A novel autoencoder based multiscale attention mechanism is incorporated with EANet that feeds on both the OCT image and edge‐enhanced OCT image at every level of the encoder. The proposed network, EANet, has been trained and tested for the segmentation of all three types of fluids on the RETOUCH challenge dataset, and the segmentation of the IRF on the OPTIMA challenge and DUKE DME datasets. The average dice coefficient of IRF, SRF, and PED for the RETOUCH dataset is 0.683, 0.873, and 0.756, respectively, whereas it is 0.805, 0.77, and 0.756 for Cirrus, Spectralis, and Topcon vendors, respectively. The proposed method outperformed all the teams that participated in the OPTIMA challenge on all types of vendor images in terms of dice coefficient. The average dice coefficients of IRF on the OPTIMA and DUKE DME datasets are 0.84 and 0.72, respectively.

Expectation-maximization algorithm leads to domain adaptation for a perineural invasion and nerve extraction task in whole slide digital pathology images.

In addition to lymphatic and vascular channels, tumor cells can also spread via nerves, i.e., perineural invasion (PNI). PNI serves as an independent prognostic indicator in many malignancies. As a result, identifying and determining the extent of PNI is an important yet extremely tedious task in surgical pathology. In this work, we present a computational approach to extract nerves and PNI from whole slide histopathology images. We make manual annotations on selected prostate cancer slides once but then apply the trained model for nerve segmentation to both prostate cancer slides and head and neck cancer slides. For the purpose of multi-domain learning/prediction and investigation on the generalization capability of deep neural network, an expectation-maximization (EM)-based domain adaptation approach is proposed to improve the segmentation performance, in particular for the head and neck cancer slides. Experiments are conducted to demonstrate the segmentation performances. The average Dice coefficient for prostate cancer slides is 0.82 and 0.79 for head and neck cancer slides. Comparisons are then made for segmentations with and without the proposed EM-based domain adaptation on prostate cancer and head and neck cancer whole slide histopathology images from The Cancer Genome Atlas (TCGA) database and significant improvements are observed.

Automatic lower limb bone segmentation in radiographs with different orientations and fields of view based on a contextual network.

Bone identification and segmentation in X-ray images are crucial in orthopedics for the automation of clinical procedures, but it often involves some manual operations. In this work, using a modified SegNet neural network, we automatically identify and segment lower limb bone structures on radiographs presenting various fields of view and different patient orientations. A wide contextual neural network architecture is proposed to perform a high-quality pixel-wise semantic segmentation on X-ray images presenting structures with a similar appearance and strong superposition. The proposed architecture is based on the premise that every output pixel on the label map has a wide receptive field. This allows the network to capture both global and local contextual information. The overlapping between structures is handled with additional labels. The proposed approach was evaluated on a test dataset composed of 70 radiographs with entire and partial bones. We obtained an average detection rate of 98.00% and an average Dice coefficient of 95.25 ± 9.02% across all classes. For the challenging subset of images with high superposition, we obtained an average detection rate of 96.36% and an average Dice coefficient of 93.81 ± 10.03% across all classes. The results show the effectiveness of the proposed approach in segmenting and identifying lower limb bone structures and overlapping structures in radiographs with strong bone superposition and highly variable configurations, as well as in radiographs containing only small pieces of bone structures.

An Automatic Segmentation Method for Lung Tumor Based on Improved Region Growing Algorithm.

In medical image processing, accurate segmentation of lung tumors is very important. Computer-aided accurate segmentation can effectively assist doctors in surgery planning and treatment decisions. Although the accurate segmentation results of lung tumors can provide a reliable basis for clinical treatment, the key to obtaining accurate segmentation results is how to improve the segmentation performance of the algorithm. We propose an automatic segmentation method for lung tumors based on an improved region growing algorithm, which uses the prior information on lung tumors to achieve an automatic selection of the initial seed point. The proposed method includes a seed point expansion mechanism and an automatic threshold update mechanism and takes the combination of multiple segmentation results as the final segmentation result. In the lung image database consortium (LIDC-IDRI) dataset, we designed 10 experiments to test the proposed method and compare it with 4 popular segmentation methods. The experimental results show that the average dice coefficient obtained by the proposed method is 0.936 ± 0.027, and the average Jaccard distance is 0.114 ± 0.049. The average dice coefficient obtained by the proposed method is 0.107, 0.053, 0.040, and 0.156, higher than that of the other four methods, respectively. This study proves that the proposed method can automatically segment lung tumors in CT slices and has suitable segmentation performance.

Abstract 12284: Utilizing Automated Machine Learning for Rheumatic Heart Disease Detection in Doppler Echocardiography

Introduction: Rheumatic heart disease (RHD) is the number one cause globally of morbidity and mortality from heart disease in children and young adults. The mitral regurgitation (MR) jet length on color Doppler echocardiography is an important index for diagnosis, but its measurement and interpretation vary. Objective: Develop an automatic machine learning approach to identify and measure the MR jet length on color Doppler for RHD detection. Methods: We used 316 echocardiograms from 95 children (mean age 12±2 years; range 5 to 17 years) with DICOM color Doppler images of the mitral valve taken from parasternal long axis (PLAX) and apical 4 chamber (AP4) views. All echocardiograms were independently reviewed by an adjudication panel consisting of four expert pediatric cardiologists to determine maximum MR jet length and diagnosis (RHD or normal). Among 95 cases, 29 were normal and 66 had RHD. Our automated method included. (1) Selection of frames during ventricular systole using a convolutional neural network architecture. (2) Localization of left atrium using convolutional neural networks with LinkNet structure. (3) Measurement of MR jet length by image color analysis. (4) Detection of RHD by applying a generalized regression model based on the maximum MR jet length measured and maximizing the balanced accuracy using cross validation. Results: Machine learning selected the correct systolic frame with an average accuracy of 0.95 (sensitivity 97%/specificity 93%) and 0.94 (sensitivity 94%/specificity 94%) for the AP4 and PLAX view, respectively. It localized the atrium with an average Dice coefficient of 0.89 and 0.9 for the AP4 and PLAX view, respectively. We estimated the MR jet length with an average absolute error of 0.33±0.4 cm (p-value = 0.15 compared to manual measurements). Our deep learning approach performed similar to or better than previously published manual methods for categorization of RHD positive vs negative. The accuracy of RHD detection was 0.84 (sensitivity 86%/specificity 79%). Conclusions: Our automatic method has the potential to reliably detect RHD as accurately as expert cardiologists. This innovative approach holds promise to scale echocardiography screening for RHD and expand prophylaxis to prevent progression of RHD globally.

A Fissure-Aided Registration Approach for Automatic Pulmonary Lobe Segmentation Using Deep Learning.

The segmentation of pulmonary lobes is important in clinical assessment, lesion location, and surgical planning. Automatic lobe segmentation is challenging, mainly due to the incomplete fissures or the morphological variation resulting from lung disease. In this work, we propose a learning-based approach that incorporates information from the local fissures, the whole lung, and priori pulmonary anatomy knowledge to separate the lobes robustly and accurately. The prior pulmonary atlas is registered to the test CT images with the aid of the detected fissures. The result of the lobe segmentation is obtained by mapping the deformation function on the lobes-annotated atlas. The proposed method is evaluated in a custom dataset with COPD. Twenty-four CT scans randomly selected from the custom dataset were segmented manually and are available to the public. The experiments showed that the average dice coefficients were 0.95, 0.90, 0.97, 0.97, and 0.97, respectively, for the right upper, right middle, right lower, left upper, and left lower lobes. Moreover, the comparison of the performance with a former learning-based segmentation approach suggests that the presented method could achieve comparable segmentation accuracy and behave more robustly in cases with morphological specificity.

An End-to-End Auto-Prediction Model Response to Neoadjuvant Chemoradiotherapy for Rectal Cancer Based on Multimodal Segmentation and Multipath Lightweight Convolution

<h3>Purpose/Objective(s)</h3> Neoadjuvant chemoradiotherapy (nCRT) plus total mesorectal excision (TME) has become the standard treatment for patients with locally advanced rectal cancer (LARC). Therefore, accurate prediction of the tumor downstaging level and pathological complete remission (pCR) has signification impacts on the further treatment and surgical decisions. Currently, the judgment mainly relies on experience of clinical experts or clinical experts combined with radiomics semi-automatically. In this study, we developed an end-to-end fully automated model of lesion segmentation and tumor downstaging and pCR prediction without manual operation. By combining pre-segmentation and post-classification modules, the work purposes to realize automatic and integrated prediction of tumor downstaging and pCR. <h3>Materials/Methods</h3> We retrospectively enrolled 350 continuous patients with LARC who received standard nCRT. In the network design section, we proposed an integrated auto-prediction model including four-channel 3D nnUNet segmentation module and multipath fusion classification module via lightweight convolution. In the auto-segmentation module, 280 patients were used for training and 70 were split for validation. Each MRI image type corresponded to a 3D nnUNet channel. For every 3D nnUNet, the numbers of downsampling in the encoder side and the decoder side were designed to be 5. And each sampling layer was connected behind every two 3D convolution modules. The total training epoch was 300 and learning rate was set to 1e-3 with during training. In the prediction module, the segmentation results of the four channels were fused into the lightweight prediction network. Adam optimizer was applied (learning rate=7.0e-6, decay=1.0e-5) and the batch size was set to 25 with 500 epoch. <h3>Results</h3> Our experimental results showed relatively stable segmentation accuracy and prediction performance. For four types of MRI images (pre and post nCRT T2-weighted, DWI), the segmentation results are as follows: (1) pre-nCRT T2-weighted MRI: The average Dice coefficient was 0.81 (median 0.84, Maximum 0.93). (2): post-nCRT T2-weighted Mirtha average Dice coefficient was 0.60(median 0.66, Maximum 0.90). (3): pre-nCRT DWI (logb1000) Mirtha average Dice coefficient was 0.73(median 0.76, Maximum 0.88). (4): post-nCRT DWI (logb1000) MRI. The average Dice coefficient was 0.42(median 0.50, Maximum 0.77). In the prediction module, the training set accuracy was 0.99 and the accuracy was 0.73 validating on 90 samples. <h3>Conclusion</h3> The entire network can achieve rectal tumor segmentation in 3D volume level and classification of pCR and tumor downstaging degree by fusing the image information of T2-weighted MRI and diffusion-weighted MRI deeply. This is a potentially valuable method for the clinical application of neoadjuvant chemoradiotherapy prediction.

FYU-Net: A Cascading Segmentation Network for Kidney Tumor Medical Imaging.

Automated segmentation of renal tumors is essential for the diagnostic evaluation of kidney cancer. However, renal tumor volume is generally small compared with the volume of the kidney and is irregularly distributed; moreover, the location and shape of renal tumors are highly variable, making the segmentation task extremely challenging. To solve the aforementioned problems, a cascaded segmentation model (FYU-Net) for computed tomography (CT) images is proposed in this paper to achieve automatic kidney tumor segmentation. The proposed model involves two main steps. In the first step, a fast scan of the kidney CT data is performed using a localization network to find slices containing tumors, and coarse segmentation is performed simultaneously. In the second step, a segmentation framework embedded with the feature pyramid network module is employed to finely segment kidney tumors. By building a feature pyramid structure, targets of different sizes are distributed to be detected on different feature layers to extract richer feature information. In addition, the top-down structure allows the information of the higher-level feature maps to be transferred to the lower-level feature maps, enhancing the semantic information of the lower-level feature maps. Comparative experiments were conducted on the Kidney PArsing Challenge 2022 public dataset; the average Jaccard coefficient and average Dice coefficient of tumor structure segmentation were more than 70.73% and more than 82.85%, respectively. The results demonstrate the effectiveness of the proposed model for kidney tumor segmentation.

Evaluation of Automated Coverage and Distance Mapping Selections to Improve Reliability and Clinical Utility of 3D Weightbearing CT Assessments

Category: Hindfoot; Basic Sciences/Biologics; Other Introduction/Purpose: Progressive collapsing foot deformity (PCFD) is a complex three-dimensional (3D) deformity where adjacent structures may adopt subtle differences in positioning that result in increased contact or subluxation. Recent studies have highlighted the need for and utility of 3D analyses in PCFD using weightbearing CT (WBCT) and bone segmentation. Beyond the limitations of triplanar imaging, 3D distance and coverage mapping analyses have further highlighted key regional differences like sinus tarsi narrowing ahead of impingement and early middle facet uncoverage ahead of collapse. However, these analyses rely upon manual identification of subregions hindering the utility of 3D mapping clinically. The objective of this study was to compare an automated selection process with manual selections in the context of subtalar regional distance and coverage maps in PCFD. Methods: In this IRB-approved retrospective study, WBCT data of 20 consecutive patients with flexible PCFD and 10 controls were analyzed. Subregions of the peritalar surface (middle and posterior facets of the calcaneus and talus; sinus tarsi area) were manually selected by two experts on manually generated bone surfaces of all 30 feet. An automated algorithm for selecting coverage area was applied to identify the same regions on the semi-automatically generated bones (Figure). A 3D distance mapping (DM) technique was used to create coverage maps (CMs) across the entire peritalar surface where areas with distances less than 4mm were defined as covered. DM and CM percentages were compared using intra-class correlations and t-tests between PCFD and control groups. The Sørensen–Dice index, or Dice coefficient, was used for comparisons of selections on the semi-automated surfaces to evaluate reproducibility of expert selections. Results: The automated process produced identical selections resulting in perfect intra-method ICCs of 1.00 for all regions and Dice coefficients of 1.00. The average Dice coefficient for all manual selections was 0.903 (range: 0.865-0.935) indicating that observers were able to reliably select the same regions with 90% overlap. When assessing reliability of manual selections, intra- observer ICCs ranged from 0.41-0.92 while inter-observer ICCs ranged from 0.47-0.99 were found. Despite strong significant correlations, average coverage was significantly lower in the sinus tarsi region of the automated selections vs the manual selections (34.3+-16.8% vs 23.1+-12.7%, p&lt;0.005). However, mean distances in each region were not significantly different in the middle facet or the sinus tarsi regions (p=0.323, p=0.095, respectively). Conclusion: Understanding of the complex 3D deformities that constitute PCFD requires sensitive and reproducible measures. Fully automated 3D assessments of coverage and bone relations may have the potential to improve understanding of these deformities aiding in diagnosis, staging, and objective evaluation of treatment effects. Prior work with 3D coverage and distance maps specifically identified the middle facet and sinus tarsi regions as being of particular importance. Compared to manual selections, these regions were well identified by the automated process. This represents a step toward the goal of viable use of fully automated 3D coverage and distance mapping when evaluating PCFD patients.

Automatic Cerebral Hemisphere Segmentation in Rat MRI with Ischemic Lesions via Attention-based Convolutional Neural Networks

We present MedicDeepLabv3+, a convolutional neural network that is the first completely automatic method to segment cerebral hemispheres in magnetic resonance (MR) volumes of rats with ischemic lesions. MedicDeepLabv3+ improves the state-of-the-art DeepLabv3+ with an advanced decoder, incorporating spatial attention layers and additional skip connections that, as we show in our experiments, lead to more precise segmentations. MedicDeepLabv3+ requires no MR image preprocessing, such as bias-field correction or registration to a template, produces segmentations in less than a second, and its GPU memory requirements can be adjusted based on the available resources. We optimized MedicDeepLabv3+ and six other state-of-the-art convolutional neural networks (DeepLabv3+, UNet, HighRes3DNet, V-Net, VoxResNet, Demon) on a heterogeneous training set comprised by MR volumes from 11 cohorts acquired at different lesion stages. Then, we evaluated the trained models and two approaches specifically designed for rodent MRI skull stripping (RATS and RBET) on a large dataset of 655 MR rat brain volumes. In our experiments, MedicDeepLabv3+ outperformed the other methods, yielding an average Dice coefficient of 0.952 and 0.944 in the brain and contralateral hemisphere regions. Additionally, we show that despite limiting the GPU memory and the training data, our MedicDeepLabv3+ also provided satisfactory segmentations. In conclusion, our method, publicly available at https://github.com/jmlipman/MedicDeepLabv3Plus, yielded excellent results in multiple scenarios, demonstrating its capability to reduce human workload in rat neuroimaging studies.

Discriminating Between Papilledema and Optic Disc Drusen Using 3D Structural Analysis of the Optic Nerve Head.

The distinction of papilledema from other optic nerve head (ONH) lesions mimicking papilledema, such as optic disc drusen (ODD), can be difficult in clinical practice. We aimed the following: (1) to develop a deep learning algorithm to automatically identify major structures of the ONH in 3-dimensional (3D) optical coherence tomography (OCT) scans and (2) to exploit such information to robustly differentiate among ODD, papilledema, and healthy ONHs. This was a cross-sectional comparative study of patients from 3 sites (Singapore, Denmark, and Australia) with confirmed ODD, those with papilledema due to raised intracranial pressure, and healthy controls. Raster scans of the ONH were acquired using OCT imaging and then processed to improve deep-tissue visibility. First, a deep learning algorithm was developed to identify major ONH tissues and ODD regions. The performance of our algorithm was assessed using the Dice coefficient. Second, a classification algorithm (random forest) was designed to perform 3-class classifications (1: ODD, 2: papilledema, and 3: healthy ONHs) strictly from their drusen and prelamina swelling scores (calculated from the segmentations). To assess performance, we reported the area under the receiver operating characteristic curve for each class. A total of 241 patients (256 imaged ONHs, including 105 ODD, 51 papilledema, and 100 healthy ONHs) were retrospectively included in this study. Using OCT images of the ONH, our segmentation algorithm was able to isolate neural and connective tissues and ODD regions/conglomerates whenever present. This was confirmed by an averaged Dice coefficient of 0.93 ± 0.03 on the test set, corresponding to good segmentation performance. Classification was achieved with high AUCs, that is, 0.99 ± 0.001 for the detection of ODD, 0.99 ± 0.005 for the detection of papilledema, and 0.98 ± 0.01 for the detection of healthy ONHs. Our artificial intelligence approach can discriminate ODD from papilledema, strictly using a single OCT scan of the ONH. Our classification performance was very good in the studied population, with the caveat that validation in a much larger population is warranted. Our approach may have the potential to establish OCT imaging as one of the mainstays of diagnostic imaging for ONH disorders in neuro-ophthalmology, in addition to fundus photography.

Impact of SSTR PET on Inter-Observer Variability of Target Delineation of Meningioma and the Possibility of Using Threshold-Based Segmentations in Radiation Oncology.

Simple SummaryDifferences in tumor segmentations between radiation oncologists is one of the largest sources of uncertainty in radiation therapy planning. This study investigated the influence of additional functional information from somatostatin receptor PET imaging on the inter observer variability in the delineation of meningioma. Further, this study assessed the usability of a simple thresholding approach for lesion delineation. It could be shown, that additional PET information was able to significantly reduce the inter observer variability. The threshold based delineation approach required a relatively low threshold value and showed only moderate agreement with the radiation oncologists.Aim: The aim of this study was to assess the effects of including somatostatin receptor agonist (SSTR) PET imaging in meningioma radiotherapy planning by means of changes in inter-observer variability (IOV). Further, the possibility of using threshold-based delineation approaches for semiautomatic tumor volume definition was assessed. Patients and Methods: Sixteen patients with meningioma undergoing fractionated radiotherapy were delineated by five radiation oncologists. IOV was calculated by comparing each delineation to a consensus delineation, based on the simultaneous truth and performance level estimation (STAPLE) algorithm. The consensus delineation was used to adapt a threshold-based delineation, based on a maximization of the mean Dice coefficient. To test the threshold-based approach, seven patients with SSTR-positive meningioma were additionally evaluated as a validation group. Results: The average Dice coefficients for delineations based on MRI alone was 0.84 ± 0.12. For delineation based on MRI + PET, a significantly higher dice coefficient of 0.87 ± 0.08 was found (p < 0.001). The Hausdorff distance decreased from 10.96 ± 11.98 mm to 8.83 ± 12.21 mm (p < 0.001) when adding PET for the lesion delineation. The best threshold value for a threshold-based delineation was found to be 14.0% of the SUVmax, with an average Dice coefficient of 0.50 ± 0.19 compared to the consensus delineation. In the validation cohort, a Dice coefficient of 0.56 ± 0.29 and a Hausdorff coefficient of 27.15 ± 21.54 mm were found for the threshold-based approach. Conclusions: SSTR-PET added to standard imaging with CT and MRI reduces the IOV in radiotherapy planning for patients with meningioma. When using a threshold-based approach for PET-based delineation of meningioma, a relatively low threshold of 14.0% of the SUVmax was found to provide the best agreement with a consensus delineation.

Intra- and interobserver assessments of intestinal wall thickness and segmentations from transverse sections of feline abdominal ultrasound images.

Measurements of intestinal wall thicknesses from ultrasound imaging (US) are routinely used to support diagnoses of intestinal disorders in cats, however published studies describing observer agreement are currently lacking. The aim of this retrospective, observer agreement study was to quantify inter- and intraobserver repeatability and agreement in the measurement of intestinal wall layer thicknesses and the segmentation of transverse sections of small intestines in US images of 20 cats. Intestinal wall layer thickness measurements of the mucosa, submucosa, muscularis, serosa layer, and total thickness of these layers were performed on five cats with small cell epitheliotropic lymphoma, five with inflammatory bowel disease, and 10 with other conditions. Thickness measurements and the segmentation encompassing the serosa layer were obtained from five observers four times non-sequentially. The average standard deviation in thickness measurements (95% confidence interval) in the mucosa, submucosa, muscularis, serosa, and total thickness were 0.35 (0.07-0.95), 0.24 (0.07-0.52), 0.22 (0.06-0.49), 0.20 (0.05-0.49), and 0.57 (0.11-1.60) mm, respectively. The average intraclass correlation coefficients, which estimates the degree of consistency in thickness measurements and segmentation areas for each observer, ranged from 0.355 to 0.870 and 0.850 to 0.993, respectively. The interclass correlation coefficient, which estimates the degree of consistency when measuring a thickness or segmentation area over all observers ranged from 0.115 to 0.753, and 0.811 to 0.902, respectively. The overall average Dice Coefficient, which estimates the extent of overlap of the segmentations for all observers was 0.957 (0.933 to 0.972). Our results suggest segmentations of small intestines have a higher interobserver agreement than measurements of intestinal wall thicknesses.

Automated irreversible electroporated region prediction using deep neural network, a preliminary study for treatment planning

ABSTRACT The primary purpose of cancer treatment with irreversible electroporation (IRE) is to maximize tumor damage and minimize surrounding healthy tissue damage. Finite element analysis is one of the popular ways to calculate electric field and cell kill probability in IRE. However, this method also has limitations. This paper will focus on using a deep neural network (DNN) in IRE to predict irreversible electroporated regions for treatment planning purposes. COMSOL Multiphysics was used to simulate the IRE. The electric conductivity change during IRE was considered to create accurate data sets of electric field distribution and cell kill probability distributions. We used eight pulses with a pulse width of 100 μs, frequency of 1 Hz, and different voltages. To create masks for DNN training, a 90% cell kill probability contour was used. After data set creation, U-Net architecture was trained to predict irreversible electroporated regions. In this study, the average U-Net DICE coefficient on test data was 0.96. Also, the average accuracy of U-Net for predicting irreversible electroporated regions was 0.97. As far as we are aware, this is the first time that U-Net was used to predict an irreversible electroporated region in IRE. The present study provides significant evidence for U-Net’s use for predicting an irreversible electroporated region in treatment planning.

Hybrid U-Net-based deep learning model for volume segmentation of lung nodules in CT images.

Accurate segmentation of the lung nodule in computed tomography images is a critical component of a computer-assisted lung cancer detection/diagnosis system. However, lung nodule segmentation is a challenging task due to the heterogeneity of nodules. This study is to develop a hybrid deep learning (H-DL) model for the segmentation of lung nodules with a wide variety of sizes, shapes, margins, and opacities. A dataset collected from Lung Image Database Consortium image collection containing 847 cases with lung nodules manually annotated by at least two radiologists with nodule diameters greater than 7mm and less than 45mm was randomly split into 683 training/validation and 164 independent test cases. The 50% consensus consolidation of radiologists' annotation was used as the reference standard for each nodule. We designed a new H-DL model combining two deep convolutional neural networks (DCNNs) with different structures as encoders to increase the learning capabilities for the segmentation of complex lung nodules. Leveraging the basic symmetric U-shaped architecture of U-Net, we redesigned two new U-shaped deep learning (U-DL) models that were expanded to six levels of convolutional layers. One U-DL model used a shallow DCNN structure containing 16 convolutional layers adapted from the VGG-19 as the encoder, and the other used a deep DCNN structure containing 200 layers adapted from DenseNet-201 as the encoder, while the same decoder with only one convolutional layer at each level was used in both U-DL models, and we referred to them as the shallow and deep U-DL models. Finally, an ensemble layer was used to combine the two U-DL models into the H-DL model. We compared the effectiveness of the H-DL, the shallow U-DL and the deep U-DL models by deploying them separately to the test set. The accuracy of volume segmentation for each nodule was evaluated by the 3D Dice coefficient and Jaccard index (JI) relative to the reference standard. For comparison, we calculated the median and minimum of the 3D Dice and JI over the individual radiologists who segmented each nodule, referred to as M-Dice, min-Dice, M-JI, and min-JI. For the 164 test cases with 327 nodules, our H-DL model achieved an average 3D Dice coefficient of 0.750 ± 0.135 and an average JI of 0.617 ± 0.159. The radiologists' average M-Dice was 0.778 ± 0.102, and the average M-JI was 0.651 ± 0.127; both were significantly higher than those achieved by the H-DL model (p<0.05). The radiologists' average min-Dice (0.685 ± 0.139) and the average min-JI (0.537 ± 0.153) were significantly lower than those achieved by the H-DL model (p<0.05). The results indicated that the H-DL model approached the average performance of radiologists and was superior to the radiologist whose manual segmentation had the min-Dice and min-JI. Moreover, the average Dice and average JI achieved by the H-DL model were significantly higher than those achieved by the individual shallow U-DL model (Dice of 0.745 ± 0.139, JI of 0.611 ± 0.161; p<0.05) or the individual deep U-DL model alone (Dice of 0.739 ± 0.145, JI of 0.604 ± 0.163; p<0.05). Our newly developed H-DL model outperformed the individual shallow or deep U-DL models. The H-DL method combining multilevel features learned by both the shallow and deep DCNNs could achieve segmentation accuracy comparable to radiologists' segmentation for nodules with wide ranges of image characteristics.

Open-source algorithm and software for computed tomography-based virtual pancreatoscopy and other applications

Pancreatoscopy plays a significant role in the diagnosis and treatment of pancreatic diseases. However, the risk of pancreatoscopy is remarkably greater than that of other endoscopic procedures, such as gastroscopy and bronchoscopy, owing to its severe invasiveness. In comparison, virtual pancreatoscopy (VP) has shown notable advantages. However, because of the low resolution of current computed tomography (CT) technology and the small diameter of the pancreatic duct, VP has limited clinical use. In this study, an optimal path algorithm and super-resolution technique are investigated for the development of an open-source software platform for VP based on 3D Slicer. The proposed segmentation of the pancreatic duct from the abdominal CT images reached an average Dice coefficient of 0.85 with a standard deviation of 0.04. Owing to the excellent segmentation performance, a fly-through visualization of both the inside and outside of the duct was successfully reconstructed, thereby demonstrating the feasibility of VP. In addition, a quantitative analysis of the wall thickness and topology of the duct provides more insight into pancreatic diseases than a fly-through visualization. The entire VP system developed in this study is available at https://github.com/gaoyi/VirtualEndoscopy.git.

Deep learning-based synthetization of real-time in-treatment 4D images using surface motion and pretreatment images: A proof-of-concept study.

To develop a deep learning model that maps body surface motion to internal anatomy deformation, which is potentially applicable to dose-free real-time 4D virtual image-guided radiotherapy based on skin surface data. Body contours were segmented out of 4DCT images. Deformable image registration algorithm was used to register the end-of-exhalation (EOE) phase to other phases. Deformation vector field was dimension-reduced to the first two principal components (PCs). A deep learning model was trained to predict the two PC scores of each phase from surface displacement. The instant deformation field can then be reconstructed, warping EOE image to obtain real-time CT image. This approach was validated on 4D XCAT phantom, the public DIR-Lab, and 4D-Lung dataset respectively, with and without simulated noise. Validation accuracy of the tumor centroid trajectory was observed as 0.04 ± 0.02mm on XCAT phantom. For the DIR-Lab dataset, 300 landmarks were annotated on the end-of-inhalation (EOI) images of each patient, and the mean displacements between their predicted and reference positions were below 2mm for all studied cases. For the 4D-Lung dataset, the average dice coefficients ± std between predicted and reference tumor contours at EOI phase were 0.835 ± 0.092 for all studied cases. A deep learning-based approach was proposed and validated to predict internal anatomy deformation from the surface motion, which is potentially applicable to on-line target navigation for accurate radiotherapy based on real-time 4D skin surface data and pretreatment images.

Cardiac Disease Classification Using Two-Dimensional Thickness and Few-Shot Learning Based on Magnetic Resonance Imaging Image Segmentation.

Cardiac cine magnetic resonance imaging (MRI) is a widely used technique for the noninvasive assessment of cardiac functions. Deep neural networks have achieved considerable progress in overcoming various challenges in cine MRI analysis. However, deep learning models cannot be used for classification because limited cine MRI data are available. To overcome this problem, features from cine image settings are derived by handcrafting and addition of other clinical features to the classical machine learning approach for ensuring the model fits the MRI device settings and image parameters required in the analysis. In this study, a novel method was proposed for classifying heart disease (cardiomyopathy patient groups) using only segmented output maps. In the encoder–decoder network, the fully convolutional EfficientNetB5-UNet was modified to perform the semantic segmentation of the MRI image slice. A two-dimensional thickness algorithm was used to combine the segmentation outputs for the 2D representation of images of the end-diastole (ED) and end-systole (ES) cardiac volumes. The thickness images were subsequently used for classification by using a few-shot model with an adaptive subspace classifier. Model performance was verified by applying the model to the 2017 MICCAI Medical Image Computing and Computer-Assisted Intervention dataset. High segmentation performance was achieved as follows: the average Dice coefficients of segmentation were 96.24% (ED) and 89.92% (ES) for the left ventricle (LV); the values for the right ventricle (RV) were 92.90% (ED) and 86.92% (ES). The values for myocardium were 88.90% (ED) and 90.48% (ES). An accuracy score of 92% was achieved in the classification of various cardiomyopathy groups without clinical features. A novel rapid analysis approach was proposed for heart disease diagnosis, especially for cardiomyopathy conditions using cine MRI based on segmented output maps.

A Hybrid Capsule Network for Automatic 3D Mandible Segmentation applied in Virtual Surgical Planning.

Automatic mandible segmentation of CT images is an essential step to achieve an accurate preoperative prediction of an intended target in three-dimensional (3D) virtual surgical planning. Segmentation of the mandible is a challenging task due to the complexity of the mandible structure, imaging artifacts, and metal implants or dental filling materials. In recent years, utilizing convolutional neural networks (CNNs) have made significant improvements in mandible segmentation. However, aggregating data at pooling layers in addition to collecting and labeling a large volume of data for training CNNs are significant issues in medical practice. We have optimized data-efficient 3D-UCaps to achieve the advantages of both the capsule network and the CNN, for accurate mandible segmentation on volumetric CT images. A novel hybrid loss function based on a weighted combination of the focal and margin loss functions is also proposed to handle the problem of voxel class imbalance. To evaluate the performance of our proposed method, a similar experiment was conducted with the 3D-UNet. All experiments are performed on the public domain database for computational anatomy (PDDCA). The proposed method and 3D-UNet achieved an average dice coefficient of 90% and 88% on the PDDCA, respectively. The results indicate that the proposed method leads to accurate mandible segmentation and outperforms the popular 3D-UNet model. It is concluded that the proposed approach is very effective as it requires more than 50% fewer parameters than the 3D-UNet.

HPM-Net: Hierarchical progressive multiscale network for liver vessel segmentation in CT images

Background and objectiveThe segmentation and visualization of liver vessels in 3D CT images are essential for computer-aided diagnosis and preoperative planning of liver diseases. Due to the irregular structure of liver vessels and image noise, accurate extraction of liver vessels is difficult. In particular, accurate segmentation of small vessels is always a challenge, as multiple single down-sampling usually results in a loss of information. MethodsIn this paper, we propose a hierarchical progressive multiscale learning network (HPM-Net) framework for liver vessel segmentation. Firstly, the hierarchical progressive multiscale learning network combines internal and external progressive learning methods to learn semantic information about liver vessels at different scales by acquiring receptive fields of different sizes. Secondly, to better capture vessel features, we propose a dual-branch progressive 3D Unet, which uses a dual-branch progressive (DBP) down-sampling strategy to reduce the loss of detailed information in the process of network down-sampling. Finally, a deep supervision mechanism is introduced into the framework and backbone network to speed up the network convergence and achieve better training of the network. ResultsWe conducted experiments on the public dataset 3Dircadb for liver vessel segmentation. The average dice coefficient and sensitivity of the proposed method reached 75.18% and 78.84%, respectively, both higher than the original network. ConclusionExperimental results show that the proposed hierarchical progressive multiscale network can accurately segment the labeled liver vessels from the CT images.