LIDC-IDRI Dataset Research Articles

Nodule-Plus R-CNN and Deep Self-Paced Active Learning for 3D Instance Segmentation of Pulmonary Nodules

Accurate and automatic segmentation of pulmonary nodules in 3D thoracic Computed Tomography (CT) images is of great significance for Computer-Aided medical Diagnosis (CAD) of lung cancer. Currently, this important task remains challenging for lack of the voxel-level annotation and training strategies that balance target/background voxels in thoracic CT images. In this paper, a new region-based network, called Nodule-plus Region-based CNN, is proposed to detect pulmonary nodules in 3D thoracic CT images effectively while synchronously generating an instance segmentation mask for every detected instance. Our new network is constructed with a stack of convolutional blocks in which lateral connections are used to alleviate the difficulty of vanishing gradients. In addition, in order to reduce annotation workload and make best use of unannotated samples, we proposed a new Deep Self-paced Active Learning (DSAL) strategy by combining Active Learning (AL) and Self-Paced Learning (SPL) strategies. For the purpose of evaluating the performance of our proposed Nodule-plus R-CNN, we conduct a series of experiments on the public LIDC-IDRI dataset, and our model achieves 0.66 Dice and 0.96 TP Dice, which are state-of-the-art best results of pulmonary nodule segmentation. When the amount of available annotated samples is limited, our model trained with the DSAL strategy performs much better than that trained with the standard strategy.

An unsupervised metaheuristic search approach for segmentation and volume measurement of pulmonary nodules in lung CT scans

This paper proposes a new methodology to automatically segment and measure the volume of pulmonary nodules in lung computed tomography (CT) scans. Estimating the malignancy likelihood of a pulmonary nodule based on lesion characteristics motivated the development of an unsupervised pulmonary nodule segmentation and volume measurement as a preliminary stage for pulmonary nodule characterization. The idea is to optimally cluster a set of feature vectors composed by intensity and shape-related features in a given feature data space extracted from a pre-detected nodule. For that purpose, a metaheuristic search based on evolutionary computation is used for clustering the corresponding feature vectors. The proposed method is simple, unsupervised and is able to segment different types of nodules in terms of location and texture without the need for any manual annotation. We validate the proposed segmentation and volume measurement on the Lung Image Database Consortium and Image Database Resource Initiative – LIDC-IDRI dataset. The first dataset is a group of 705 solid and sub-solid (assessed as part-solid and non-solid) nodules located in different regions of the lungs, and the second, more challenging, is a group of 59 sub-solid nodules. The average Dice scores of 82.35% and 71.05% for the two datasets show the good performance of the segmentation proposal. Comparisons with previous state-of-the-art techniques also show acceptable and comparable segmentation results. The volumes of the segmented nodules are measured via ellipsoid approximation. The correlation and statistical significance between the measured volumes of the segmented nodules and the ground-truth are obtained by Pearson correlation coefficient value, obtaining an R-value ≥ 92.16% with a significance level of 5%.

Knowledge-based Collaborative Deep Learning for Benign-Malignant Lung Nodule Classification on Chest CT.

The accurate identification of malignant lung nodules on chest CT is critical for the early detection of lung cancer, which also offers patients the best chance of cure. Deep learning methods have recently been successfully introduced to computer vision problems, although substantial challenges remain in the detection of malignant nodules due to the lack of large training data sets. In this paper, we propose a multi-view knowledge-based collaborative (MV-KBC) deep model to separate malignant from benign nodules using limited chest CT data. Our model learns 3-D lung nodule characteristics by decomposing a 3-D nodule into nine fixed views. For each view, we construct a knowledge-based collaborative (KBC) submodel, where three types of image patches are designed to fine-tune three pre-trained ResNet-50 networks that characterize the nodules' overall appearance, voxel, and shape heterogeneity, respectively. We jointly use the nine KBC submodels to classify lung nodules with an adaptive weighting scheme learned during the error back propagation, which enables the MV-KBC model to be trained in an end-to-end manner. The penalty loss function is used for better reduction of the false negative rate with a minimal effect on the overall performance of the MV-KBC model. We tested our method on the benchmark LIDC-IDRI data set and compared it to the five state-of-the-art classification approaches. Our results show that the MV-KBC model achieved an accuracy of 91.60% for lung nodule classification with an AUC of 95.70%. These results are markedly superior to the state-of-the-art approaches.

Analysis of Lung Tumour Detection and Segmentation Using Level Set Method of Active Contour Model

Lung cancer is the most common leading cancer in both men and women all over the world. Accurate image segmentation is an essential image analysis tool that is responsible for partitioning an image into several sub-regions. Active contour model have been widely used for effective image segmentation methods as this model produce sub-regions with continuous boundaries. It is used in the applications such as image analysis, deep learning, computer vision and machine learning. Advanced level set method helps to implement active contours for image segmentation with good boundary detection accuracy. This paper proposes a model based on active contour using level set methods for segmentation of such lung CT images and focusing 3D lesion refinement. The features were determined by applying a multi-scale Gaussian filter. This proposed method is able to detect lung tumors in CT images with intensity, homogeneity and noise. The proposed method uses LIDC-IDRI dataset images to segment accurate 3D lesion of lung tumor CT images.

3D deep learning for detecting pulmonary nodules in CT scans.

To demonstrate and test the validity of a novel deep-learning-based system for the automated detection of pulmonary nodules. The proposed system uses 2 3D deep learning models, 1 for each of the essential tasks of computer-aided nodule detection: candidate generation and false positive reduction. A total of 888 scans from the LIDC-IDRI dataset were used for training and evaluation. Results for candidate generation on the test data indicated a detection rate of 94.77% with 30.39 false positives per scan, while the test results for false positive reduction exhibited a sensitivity of 94.21% with 1.789 false positives per scan. The overall system detection rate on the test data was 89.29% with 1.789 false positives per scan. An extensive and rigorous validation was conducted to assess the performance of the proposed system. The system demonstrated a novel combination of 3D deep neural network architectures and demonstrates the use of deep learning for both candidate generation and false positive reduction to be evaluated with a substantial test dataset. The results strongly support the ability of deep learning pulmonary nodule detection systems to generalize to unseen data. The source code and trained model weights have been made available. A novel deep-neural-network-based pulmonary nodule detection system is demonstrated and validated. The results provide comparison of the proposed deep-learning-based system over other similar systems based on performance.

Abstract 3048: Deep learning to predict survival prognosis for patients with non-small cell lung cancer using images and clinical data

Abstract Goal We present a deep learning framework to predict survival prognosis of patients with non-small cell lung cancer using CT images and clinical data developed and validated in multi-institutional cohorts. Methods We developed 3D convolutional neural networks (CNNs) using CT images, tumor nodule segmentation, clinical data (i.e. age, sex, histology, and stage) and survival times as input to predict survival of non-small cell lung cancer. Both the CT images and the segmentations were used as inputs for the CNN, the latter to localize the volumes-of-interests (VOIs) in the CT scans. We used three cohorts to evaluate our framework: Moffitt (n=186), Maastro (n=311), and Stanford (n=130). We investigated pre-training with the LIDC-IDRI dataset (n=1010) and compared deep learning performance with 1674 manually-curated radiomics features and clinical data. We used two ways to evaluate the models. First, we split the three cohorts into training and test sets by selecting two of the cohorts for training and the remaining for testing. Secondly, we combined all cohorts and performed a 10-fold stratified cross-validation. We used the Concordance Index (CI) to estimate model performance. Next, we evaluated priming whereby we use 10% of the validation cohort during training and test on the remaining patients in the test cohort. To ensure that the test set was identical for both priming and non-priming experiments, we discarded these samples for the non-priming tests. Results Table 1 shows the results of the two evaluation strategies. We observed consistently high CIs for all three cohorts. LIDC pretraining and priming yielded the highest CIs as compared to radiomic and clinical-only pipelines. For the 10-fold cross-validation experiment, our pipeline achieved an average CI=0.64. Conclusion Our study highlights the promising results of our pipeline to predict survival prognosis across three cohorts without the need to define radiomics features. Train on two cohorts, test on 3rd cohortpretrain on LIDC-IDRItest w/o primingtest w/ primingtest w/o priming (only radiomics features / clinical)test w/ priming (only radiomics features / clinical)train on Moffitt + Maastro, test on Stanfordyes0.620.630.52 (radiomics only)0.53 (radiomics only)train on Moffitt + Maastro, test on Stanfordno0.610.620.57 (clinical only)0.57 (clinical only)train on Moffitt + Stanford, test on Maastroyes0.60.60.55 (radiomics only)0.55 (radiomics only)train on Moffitt + Stanford, test on Maastrono0.580.60.56 (clinical only)0.56 (clinical only)train on Stanford + Maastro, test on Moffittyes0.580.60.53 (radiomics only)0.53 (radiomics only)train on Stanford + Maastro, test on Moffittno0.60.570.56 (clinical only)0.56 (clinical only) Citation Format: Edward H. Lee, Mu Zhou, Noah Gamboa, Kevin Brennan, Haruka Itakura, Viswam Nair, Sandy Napel, Simon Wong, Olivier Gevaert. Deep learning to predict survival prognosis for patients with non-small cell lung cancer using images and clinical data [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2018; 2018 Apr 14-18; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2018;78(13 Suppl):Abstract nr 3048.

Automatic recognition of 3D GGO CT imaging signs through the fusion of hybrid resampling and layer-wise fine-tuning CNNs.

Ground-glass opacity (GGO) is a common CT imaging sign on high-resolution CT, which means the lesion is more likely to be malignant compared to common solid lung nodules. The automatic recognition of GGO CT imaging signs is of great importance for early diagnosis and possible cure of lung cancers. The present GGO recognition methods employ traditional low-level features and system performance improves slowly. Considering the high-performance of CNN model in computer vision field, we proposed an automatic recognition method of 3D GGO CT imaging signs through the fusion of hybrid resampling and layer-wise fine-tuning CNN models in this paper. Our hybrid resampling is performed on multi-views and multi-receptive fields, which reduces the risk of missing small or large GGOs by adopting representative sampling panels and processing GGOs with multiple scales simultaneously. The layer-wise fine-tuning strategy has the ability to obtain the optimal fine-tuning model. Multi-CNN models fusion strategy obtains better performance than any single trained model. We evaluated our method on the GGO nodule samples in publicly available LIDC-IDRI dataset of chest CT scans. The experimental results show that our method yields excellent results with 96.64% sensitivity, 71.43% specificity, and 0.83 F1 score. Our method is a promising approach to apply deep learning method to computer-aided analysis of specific CT imaging signs with insufficient labeled images. Graphical abstract We proposed an automatic recognition method of 3D GGO CT imaging signs through the fusion of hybrid resampling and layer-wise fine-tuning CNN models in this paper. Our hybrid resampling reduces the risk of missing small or large GGOs by adopting representative sampling panels and processing GGOs with multiple scales simultaneously. The layer-wise fine-tuning strategy has ability to obtain the optimal fine-tuning model. Our method is a promising approach to apply deep learning method to computer-aided analysis of specific CT imaging signs with insufficient labeled images.

Fusing texture, shape and deep model-learned information at decision level for automated classification of lung nodules on chest CT

The separation of malignant from benign lung nodules on chest computed tomography (CT) is important for the early detection of lung cancer, since early detection and management offer the best chance for cure. Although deep learning methods have recently produced a marked improvement in image classification there are still challenges as these methods contain myriad parameters and require large-scale training sets that are not usually available for most routine medical imaging studies. In this paper, we propose an algorithm for lung nodule classification that fuses the texture, shape and deep model-learned information (Fuse-TSD) at the decision level. This algorithm employs a gray level co-occurrence matrix (GLCM)-based texture descriptor, a Fourier shape descriptor to characterize the heterogeneity of nodules and a deep convolutional neural network (DCNN) to automatically learn the feature representation of nodules on a slice-by-slice basis. It trains an AdaBoosted back propagation neural network (BPNN) using each feature type and fuses the decisions made by three classifiers to differentiate nodules. We evaluated this algorithm against three approaches on the LIDC-IDRI dataset. When the nodules with a composite malignancy rate 3 were discarded, regarded as benign or regarded as malignant, our Fuse-TSD algorithm achieved an AUC of 96.65%, 94.45% and 81.24%, respectively, which was substantially higher than the AUC obtained by other approaches.

Discovery radiomics via evolutionary deep radiomic sequencer discovery for pathologically proven lung cancer detection.

While lung cancer is the second most diagnosed form of cancer in men and women, a sufficiently early diagnosis can be pivotal in patient survival rates. Imaging-based, or radiomics-driven, detection methods have been developed to aid diagnosticians, but largely rely on hand-crafted features that may not fully encapsulate the differences between cancerous and healthy tissue. Recently, the concept of discovery radiomics was introduced, where custom abstract features are discovered from readily available imaging data. We propose an evolutionary deep radiomic sequencer discovery approach based on evolutionary deep intelligence. Motivated by patient privacy concerns and the idea of operational artificial intelligence, the evolutionary deep radiomic sequencer discovery approach organically evolves increasingly more efficient deep radiomic sequencers that produce significantly more compact yet similarly descriptive radiomic sequences over multiple generations. As a result, this framework improves operational efficiency and enables diagnosis to be run locally at the radiologist's computer while maintaining detection accuracy. We evaluated the evolved deep radiomic sequencer (EDRS) discovered via the proposed evolutionary deep radiomic sequencer discovery framework against state-of-the-art radiomics-driven and discovery radiomics methods using clinical lung CT data with pathologically proven diagnostic data from the LIDC-IDRI dataset. The EDRS shows improved sensitivity (93.42%), specificity (82.39%), and diagnostic accuracy (88.78%) relative to previous radiomics approaches.

A multi-view deep convolutional neural networks for lung nodule segmentation.

We present a multi-view convolutional neural networks (MV-CNN) for lung nodule segmentation. The MV-CNN specialized in capturing a diverse set of nodule-sensitive features from axial, coronal and sagittal views in CT images simultaneously. The proposed network architecture consists of three CNN branches, where each branch includes seven stacked layers and takes multi-scale nodule patches as input. The three CNN branches are then integrated with a fully connected layer to predict whether the patch center voxel belongs to the nodule. The proposed method has been evaluated on 893 nodules from the public LIDC-IDRI dataset, where ground-truth annotations and CT imaging data were provided. We showed that MV-CNN demonstrated encouraging performance for segmenting various type of nodules including juxta-pleural, cavitary, and non-solid nodules, achieving an average dice similarity coefficient (DSC) of 77.67% and average surface distance (ASD) of 0.24, outperforming conventional image segmentation approaches.

A CADe system for nodule detection in thoracic CT images based on artificial neural network

Lung cancer has been the leading cause of cancer-related deaths in 2015 in United States. Early detection of lung nodules will undoubtedly increase the five-year survival rate for lung cancer according to prior studies. In this paper, we propose a novel rating method based on geometrical and statistical features to extract initial nodule candidates and an artificial neural network approach to the detection of lung nodules. The novel method is solely based on 3D distribution of neighboring voxels instead of user-specified features. During initial candidates detection, we combine organized region properties calculated from connected component analysis with corresponding voxel value distributions from statistical analysis to reduce false positives while retaining true nodules. Then we devise multiple artificial neural networks (ANNs) trained from massive voxel neighbor sampling of different types of nodules and organize the outputs using a 3D scoring method to identify final nodules. The experiments on 107 CT cases with 252 nodules in LIDC-IDRI data sets have shown that our new method achieves sensitivity of 89.4% while reducing the false positives to 2.0 per case. Our comprehensive experiments have demonstrated our system would be of great assistance for diagnosis of lung nodules in clinical treatments.

Discriminative Localization in CNNs for Weakly-Supervised Segmentation of Pulmonary Nodules.

Automated detection and segmentation of pulmonary nodules on lung computed tomography (CT) scans can facilitate early lung cancer diagnosis. Existing supervised approaches for automated nodule segmentation on CT scans require voxel-based annotations for training, which are labor- and time-consuming to obtain. In this work, we propose a weakly-supervised method that generates accurate voxel-level nodule segmentation trained with image-level labels only. By adapting a convolutional neural network (CNN) trained for image classification, our proposed method learns discriminative regions from the activation maps of convolution units at different scales, and identifies the true nodule location with a novel candidate-screening framework. Experimental results on the public LIDC-IDRI dataset demonstrate that, our weakly-supervised nodule segmentation framework achieves competitive performance compared to a fully-supervised CNN-based segmentation method.

Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks.

We propose a novel Computer-Aided Detection (CAD) system for pulmonary nodules using multi-view convolutional networks (ConvNets), for which discriminative features are automatically learnt from the training data. The network is fed with nodule candidates obtained by combining three candidate detectors specifically designed for solid, subsolid, and large nodules. For each candidate, a set of 2-D patches from differently oriented planes is extracted. The proposed architecture comprises multiple streams of 2-D ConvNets, for which the outputs are combined using a dedicated fusion method to get the final classification. Data augmentation and dropout are applied to avoid overfitting. On 888 scans of the publicly available LIDC-IDRI dataset, our method reaches high detection sensitivities of 85.4% and 90.1% at 1 and 4 false positives per scan, respectively. An additional evaluation on independent datasets from the ANODE09 challenge and DLCST is performed. We showed that the proposed multi-view ConvNets is highly suited to be used for false positive reduction of a CAD system.

Lung Lesion Extraction Using a Toboggan Based Growing Automatic Segmentation Approach.

The accurate segmentation of lung lesions from computed tomography (CT) scans is important for lung cancer research and can offer valuable information for clinical diagnosis and treatment. However, it is challenging to achieve a fully automatic lesion detection and segmentation with acceptable accuracy due to the heterogeneity of lung lesions. Here, we propose a novel toboggan based growing automatic segmentation approach (TBGA) with a three-step framework, which are automatic initial seed point selection, multi-constraints 3D lesion extraction and the final lesion refinement. The new approach does not require any human interaction or training dataset for lesion detection, yet it can provide a high lesion detection sensitivity (96.35%) and a comparable segmentation accuracy with manual segmentation (P > 0.05), which was proved by a series assessments using the LIDC-IDRI dataset (850 lesions) and in-house clinical dataset (121 lesions). We also compared TBGA with commonly used level set and skeleton graph cut methods, respectively. The results indicated a significant improvement of segmentation accuracy . Furthermore, the average time consumption for one lesion segmentation was under 8 s using our new method. In conclusion, we believe that the novel TBGA can achieve robust, efficient and accurate lung lesion segmentation in CT images automatically.

Information Ethics: Privacy and Intellectual Property. Lee Freeman and A. Graham Peace, Editors. Hershey, PA: Information Science Publishing, 2005. xv, 276 pp. $89.95 (paper), ISBN: 1-59140-492-4.

Spiculations are important predictors of lung cancer malignancy, which are spikes on the surface of the pulmonary nodules. In this study, we proposed an interpretable and parameter-free technique to quantify the spiculation using area distortion metric obtained by the conformal (angle-preserving) spherical parameterization. We exploit the insight that for an angle-preserved spherical mapping of a given nodule, the corresponding negative area distortion precisely characterizes the spiculations on that nodule. We introduced novel spiculation scores based on the area distortion metric and spiculation measures. We also semi-automatically segment lung nodule (for reproducibility) as well as vessel and wall attachment to differentiate the real spiculations from lobulation and attachment. A simple pathological malignancy prediction model is also introduced. We used the publicly-available LIDC-IDRI dataset pathologists (strong-label) and radiologists (weak-label) ratings to train and test radiomics models containing this feature, and then externally validate the models. We achieved AUC = 0.80 and 0.76, respectively, with the models trained on the 811 weakly-labeled LIDC datasets and tested on the 72 strongly-labeled LIDC and 73 LUNGx datasets; the previous best model for LUNGx had AUC = 0.68. The number-of-spiculations feature was found to be highly correlated (Spearman’s rank correlation coefficient ρ=0.44) with the radiologists’ spiculation score. We developed a reproducible and interpretable, parameter-free technique for quantifying spiculations on nodules. The spiculation quantification measures was then applied to the radiomics framework for pathological malignancy prediction with reproducible semi-automatic segmentation of nodule. Using our interpretable features (size, attachment, spiculation, lobulation), we were able to achieve higher performance than previous models. In the future, we will exhaustively test our model for lung cancer screening in the clinic.