3D Convolutional Neural Network Model Research Articles

Knee Osteoarthritis Classification Using 3D CNN and MRI

Osteoarthritis (OA) is the most common form of arthritis and can often occur in the knee. While convolutional neural networks (CNNs) have been widely used to study medical images, the application of a 3-dimensional (3D) CNN in knee OA diagnosis is limited. This study utilizes a 3D CNN model to analyze sequences of knee magnetic resonance (MR) images to perform knee OA classification. An advantage of using 3D CNNs is the ability to analyze the whole sequence of 3D MR images as a single unit as opposed to a traditional 2D CNN, which examines one image at a time. Therefore, 3D features could be extracted from adjacent slices, which may not be detectable from a single 2D image. The input data for each knee were a sequence of double-echo steady-state (DESS) MR images, and each knee was labeled by the Kellgren and Lawrence (KL) grade of severity at levels 0–4. In addition to the 5-category KL grade classification, we further examined a 2-category classification that distinguishes non-OA (KL ≤ 1) from OA (KL ≥ 2) knees. Clinically, diagnosing a patient with knee OA is the ultimate goal of assigning a KL grade. On a dataset with 1100 knees, the 3D CNN model that classifies knees with and without OA achieved an accuracy of 86.5% on the validation set and 83.0% on the testing set. We further conducted a comparative study between MRI and X-ray. Compared with a CNN model using X-ray images trained from the same group of patients, the proposed 3D model with MR images achieved higher accuracy in both the 5-category classification (54.0% vs. 50.0%) and the 2-category classification (83.0% vs. 77.0%). The result indicates that MRI, with the application of a 3D CNN model, has greater potential to improve diagnosis accuracy for knee OA clinically than the currently used X-ray methods.

TsanKit: artificial intelligence for solder ball head-in-pillow defect inspection

In this paper, we propose an AI (Artificial Intelligence) solution for solder ball HIP (Head-In-Pillow) defect inspection. The HIP defect will affect the conductivity of the solder balls leading to intermittent failures. Due to the variable location and shape of the HIP defect, traditional machine vision algorithms cannot solve the problem completely. In recent years, Convolutional Neural Network (CNN) has an outstanding performance in image recognition and classification, but it is easy to cause overfitting problems due to insufficient data. Therefore, we combine CNN and the machine learning algorithm Support Vector Machine (SVM) to design our inspection process. Referring to the advantages of several state-of-the-art models, we propose our 3D CNN model and adopt focal loss as well as triplet loss to solve the data imbalance problem caused by rare defective data. Our inspection method has the best performance and fast testing speed compared with several classic CNN models and the deep learning inspection software SuaKIT.

Two‐stage deep learning model for fully automated pancreas segmentation on computed tomography: Comparison with intra‐reader and inter‐reader reliability at full and reduced radiation dose on an external dataset

To develop a two-stage three-dimensional (3D) convolutional neural networks (CNNs) for fully automated volumetric segmentation of pancreas on computed tomography (CT) and to further evaluate its performance in the context of intra-reader and inter-reader reliability at full dose and reduced radiation dose CTs on a public dataset. A dataset of 1994 abdomen CT scans (portal venous phase, slice thickness≤3.75-mm, multiple CT vendors) was curated by two radiologists (R1 and R2) to exclude cases with pancreatic pathology, suboptimal image quality, and image artifacts (n=77). Remaining 1917 CTs were equally allocated between R1 and R2 for volumetric pancreas segmentation [ground truth (GT)]. This internal dataset was randomly divided into training (n=1380), validation (n=248), and test (n=289) sets for the development of a two-stage 3D CNN model based on a modified U-net architecture for automated volumetric pancreas segmentation. Model's performance for pancreas segmentation and the differences in model-predicted pancreatic volumes vs GT volumes were compared on the test set. Subsequently, an external dataset from The Cancer Imaging Archive (TCIA) that had CT scans acquired at standard radiation dose and same scans reconstructed at a simulated 25% radiation dose was curated (n=41). Volumetric pancreas segmentation was done on this TCIA dataset by R1 and R2 independently on the full dose and then at the reduced radiation dose CT images. Intra-reader and inter-reader reliability, model's segmentation performance, and reliability between model-predicted pancreatic volumes at full vs reduced dose were measured. Finally, model's performance was tested on the benchmarking National Institute of Health (NIH)-Pancreas CT (PCT) dataset. Three-dimensional CNN had mean (SD) Dice similarity coefficient (DSC): 0.91 (0.03) and average Hausdorff distance of 0.15 (0.09)mm on the test set. Model's performance was equivalent between males and females (P=0.08) and across different CT slice thicknesses (P>0.05) based on noninferiority statistical testing. There was no difference in model-predicted and GT pancreatic volumes [mean predicted volume 99cc (31cc); GT volume 101cc (33cc), P=0.33]. Mean pancreatic volume difference was -2.7cc (percent difference: -2.4% of GT volume) with excellent correlation between model-predicted and GT volumes [concordance correlation coefficient (CCC)=0.97]. In the external TCIA dataset, the model had higher reliability than R1 and R2 on full vs reduced dose CT scans [model mean (SD) DSC: 0.96 (0.02), CCC=0.995 vs R1 DSC: 0.83 (0.07), CCC=0.89, and R2 DSC:0.87 (0.04), CCC=0.97]. The DSC and volume concordance correlations for R1 vs R2 (inter-reader reliability) were 0.85 (0.07), CCC=0.90 at full dose and 0.83 (0.07), CCC=0.96 at reduced dose datasets. There was good reliability between model and R1 at both full and reduced dose CT [full dose: DSC: 0.81 (0.07), CCC=0.83 and reduced dose DSC:0.81 (0.08), CCC=0.87]. Likewise, there was good reliability between model and R2 at both full and reduced dose CT [full dose: DSC: 0.84 (0.05), CCC=0.89 and reduced dose DSC:0.83(0.06), CCC=0.89]. There was no difference in model-predicted and GT pancreatic volume in TCIA dataset (mean predicted volume 96cc (33); GT pancreatic volume 89cc (30), p=0.31). Model had mean (SD) DSC: 0.89 (0.04) (minimum-maximum DSC: 0.79 -0.96) on the NIH-PCT dataset. A 3D CNN developed on the largest dataset of CTs is accurate for fully automated volumetric pancreas segmentation and is generalizable across a wide range of CT slice thicknesses, radiation dose, and patient gender. This 3D CNN offers a scalable tool to leverage biomarkers from pancreas morphometrics and radiomics for pancreatic diseases including for early pancreatic cancer detection.

AR3D: Attention Residual 3D Network for Human Action Recognition.

At present, in the field of video-based human action recognition, deep neural networks are mainly divided into two branches: the 2D convolutional neural network (CNN) and 3D CNN. However, 2D CNN’s temporal and spatial feature extraction processes are independent of each other, which means that it is easy to ignore the internal connection, affecting the performance of recognition. Although 3D CNN can extract the temporal and spatial features of the video sequence at the same time, the parameters of the 3D model increase exponentially, resulting in the model being difficult to train and transfer. To solve this problem, this article is based on 3D CNN combined with a residual structure and attention mechanism to improve the existing 3D CNN model, and we propose two types of human action recognition models (the Residual 3D Network (R3D) and Attention Residual 3D Network (AR3D)). Firstly, in this article, we propose a shallow feature extraction module and improve the ordinary 3D residual structure, which reduces the parameters and strengthens the extraction of temporal features. Secondly, we explore the application of the attention mechanism in human action recognition and design a 3D spatio-temporal attention mechanism module to strengthen the extraction of global features of human action. Finally, in order to make full use of the residual structure and attention mechanism, an Attention Residual 3D Network (AR3D) is proposed, and its two fusion strategies and corresponding model structure (AR3D_V1, AR3D_V2) are introduced in detail. Experiments show that the fused structure shows different degrees of performance improvement compared to a single structure.

Through-Wall Human Pose Reconstruction via UWB MIMO Radar and 3D CNN

Human pose reconstruction has been a fundamental research in computer vision. However, existing pose reconstruction methods suffer from the problem of wall occlusion that cannot be solved by a traditional optical sensor. This article studies a novel human target pose reconstruction framework using low-frequency ultra-wideband (UWB) multiple-input multiple-output (MIMO) radar and a convolutional neural network (CNN), which is used to detect targets behind the wall. In the proposed framework, first, we use UWB MIMO radar to capture the human body information. Then, target detection and tracking are used to lock the target position, and the back-projection algorithm is adopted to construct three-dimensional (3D) images. Finally, we take the processed 3D image as input to reconstruct the 3D pose of the human target via the designed 3D CNN model. Field detection experiments and comparison results show that the proposed framework can achieve pose reconstruction of human targets behind a wall, which indicates that our research can make up for the shortcomings of optical sensors and significantly expands the application of the UWB MIMO radar system.

Deep Learning for Detection of Intracranial Aneurysms From Computed Tomography Angiography Images

Background: The rates of the fatality and disability will be extremely high once the intracranial aneurysms rupture. Radiologic examination based on the computed tomography angiography (CTA) images is the preferred method for the diagnosis of intracranial aneurysms, where the interpretation of imaging data is usually performed by an experienced radiologist. However, the accuracy of the interpretation and the time spent highly rely on the professional competence of the radiologist, which may yield inconsistent diagnostic outcomes. In addition, with steadily increasing workloads of radiology departments for providing various imaging diagnostic services, the fatigue of radiologists due to overloading may cause them neglecting or mistaking interpretations. Therefore, the aim of this study was to develop a new method to automatically detect intracranial aneurysms from CTA images using deep learning based on convolutional neural network (CNN) implemented on the Deepmedic platform. This new method can serve as a clinical diagnostic tool or a preliminary reference for diagnosis of intracranial aneurysms. It will be of great clinical value in reducing the diagnosis time and the missed or mistaken diagnosis caused by radiologists' different professional levels or their fatigue. Methods: We implemented a deep learning architecture with a 3D CNN model on Deepmedic platform for automatic segmentation and detection of intracranial aneurysms from CTA images which were retrospectively collected. The retrospective dataset was preprocessed and then was divided into two parts: the training part (80 subjects) to train the CNN model, and the testing part (10 subjects) to evaluate the performance of the automatic detection system. Sensitivity, positive predictive value (PPV) and the false positives per case were evaluated with the reference standard of digital subtraction angiography (DSA) or intraoperative results. Findings: The overall sensitivity and PPV of this system for the detection of intracranial aneurysms from CTA images were 92.3% and 100%, and the segmentation sensitivity also was 92.3%. The performance of the system in the detection of intracranial aneurysms was related to their size: the detection sensitivity for tiny intracranial aneurysms with a diameter ≤ 3mm was 66.7%, whereas the sensitivities of detection for large and medium-sized intracranial aneurysms of > 3mm both were 100%. Interpretation: The deep learning architecture with a 3D CNN model on Deepmedic platform could reliably segment and detect intracranial aneurysms from CTA images with an extremely high sensitivity, especially for large and medium-sized intracranial aneurysms, and this required only very limited training samples. Moreover, the detection sensitivity of tiny intracranial aneurysms was higher than those reported recently in other studies. Funding Statement: This study was supported in part by National Natural Science Foundation of China (81601558), Breeding Foundation of Zhuhai People's Hospital of China (2019PY-16) and Medical Research Foundation of Zhuhai City of China (20191207A010017). Declaration of Interests: No authors report any financial relationships with commercial interests Ethics Approval Statement: This retrospective study was approved by the Ethics Committee of Zhuhai People’s Hospital and the First Affiliated Hospital of Harbin Medical University.

Classification of Benign and Malignant Pulmonary Nodules Based on the Multiresolution 3D DPSECN Model and Semisupervised Clustering

Deep learning model training requires a large number of labeled samples, but the acquisition of labeled samples is time-consuming and laborious in the medical field. To solve this problem, a semisupervised clustering algorithm combined with a 3D convolutional neural network model is proposed to improve the classification performance for benign and malignant pulmonary nodules. The research contents are as follows: Firstly, a multiresolution 3D dual path squeeze excitation deep learning network model is constructed. Then, the feature extractor in the network model is used to extract the high-level features of the image, and semisupervised clustering is applied to the extracted image features. The corresponding pseudolabels can be obtained for the unlabeled samples, and the categories of unlabeled samples are determined and utilized. Finally, the oversampling algorithm is used to balance the data categories of different types of samples, and the benign and malignant pulmonary nodules are classified by a classifier constructed by a 3D dual path squeeze excitation network. The experimental results show that the proposed semisupervised clustering algorithm can label the categories of unlabeled samples. The proposed network model can learn more characteristics related to pulmonary nodules and can effectively improve the classification performance of pulmonary nodules. The proposed network model was tested using Lung Image Database Consortium (LIDC-IDRI) dataset, and an accuracy of 94.4% and an AUC of 0.931 were obtained. Compared with some existing classification models, the proposed method can achieve a better classification effect of pulmonary nodules.

A Fast and Compact 3-D CNN for Hyperspectral Image Classification

Hyperspectral imaging (HSI) has been extensively utilized for a number of real-world applications. HSI classification (HSIC) is a challenging task due to high inter-class similarity, high intra-class variability, overlapping, and nested regions. A 2D Convolutional Neural Network (CNN) is a viable approach whereby HSIC highly depends on both Spectral-Spatial information, therefore, 3D CNN can be an alternative but highly computational complex due to the volume and spectral dimensions. Furthermore, these models do not extract quality feature maps and may underperform over the regions having similar textures. Therefore, this work proposed a 3D CNN model that utilizes both spatial-spectral feature maps to attain good performance. In order to achieve the said performance, the HSI cube is first divided into small overlapping 3D patches. Later these patches are processed to generate 3D feature maps using a 3D kernel function over multiple contiguous bands that persevere the spectral information as well. Benchmark HSI datasets (Pavia University, Salinas and Indian Pines) are considered to validate the performance of our proposed method. The results are further compared with several state-of-the-art methods.

3D Compressed Convolutional Neural Network Differentiates Neuromyelitis Optical Spectrum Disorders From Multiple Sclerosis Using Automated White Matter Hyperintensities Segmentations.

BackgroundMagnetic resonance imaging (MRI) has a wide range of applications in medical imaging. Recently, studies based on deep learning algorithms have demonstrated powerful processing capabilities for medical imaging data. Previous studies have mostly focused on common diseases that usually have large scales of datasets and centralized the lesions in the brain. In this paper, we used deep learning models to process MRI images to differentiate the rare neuromyelitis optical spectrum disorder (NMOSD) from multiple sclerosis (MS) automatically, which are characterized by scattered and overlapping lesions.MethodsWe proposed a novel model structure to capture 3D MRI images’ essential information and converted them into lower dimensions. To empirically prove the efficiency of our model, firstly, we used a conventional 3-dimensional (3D) model to classify the T2-weighted fluid-attenuated inversion recovery (T2-FLAIR) images and proved that the traditional 3D convolutional neural network (CNN) models lack the learning capacity to distinguish between NMOSD and MS. Then, we compressed the 3D T2-FLAIR images by a two-view compression block to apply two different depths (18 and 34 layers) of 2D models for disease diagnosis and also applied transfer learning by pre-training our model on ImageNet dataset.ResultsWe found that our models possess superior performance when our models were pre-trained on ImageNet dataset, in which the models’ average accuracies of 34 layers model and 18 layers model were 0.75 and 0.725, sensitivities were 0.707 and 0.708, and specificities were 0.759 and 0.719, respectively. Meanwhile, the traditional 3D CNN models lacked the learning capacity to distinguish between NMOSD and MS.ConclusionThe novel CNN model we proposed could automatically differentiate the rare NMOSD from MS, especially, our model showed better performance than traditional3D CNN models. It indicated that our 3D compressed CNN models are applicable in handling diseases with small-scale datasets and possess overlapping and scattered lesions.

Modeling Texture in Deep 3D CNN for Survival Analysis.

Radiomics has shown remarkable potential for predicting the survival outcome for various types of cancers such as pancreatic ductal adenocarcinoma (PDAC). However, to date, there has been limited research using convolutional neural networks (CNN) with radiomic methods for this task, due to their requirement for large training sets. To overcome this issue, we propose a new type of radiomic descriptor modeling the distribution of learned features with a Gaussian mixture model (GMM). These parametric features (GMM-CNN) are computed from gross tumor volumes of PDAC patients defined semi-automatically in pre-operative computed tomography (CT) scans. We use the proposed GMM-CNN features as input to a robust classifier based on random forests (RF) to predict the survival outcome of patients with PDAC. Our experiments assess the advantage of GMM-CNN compared to employing the same 3D CNN model directly, standard radiomic (i.e., histogram, texture and shape), conditional entropy (CENT) based on 3DCNN, and clinical features (i.e., serum carbohydrate antigen 19-9 and chemotherapy neoadjuvant). Using the RF model (100 samples for training; 59 samples for validation), GMM-CNN features provided the highest area under the ROC curve (AUC) of 72.0% (p= 6.4×10-5) compared to 64.0% (p=0.01) for the 3D CNN model output, 66.8% (p=0.01) for standard radiomic features, 64.2% (p=0.003) for CENT, and 57.6% (p=0.3) for clinical variables. Our results suggest that the proposed GMM-CNN features used with a RF classifier can significantly improve the capacity to prognosticate PDAC patients prior to surgery via routinely-acquired imaging data.

Three-dimensional MRI Bone Models of the Glenohumeral Joint Using Deep Learning: Evaluation of Normal Anatomy and Glenoid Bone Loss.

To use convolutional neural networks (CNNs) for fully automated MRI segmentation of the glenohumeral joint and evaluate the accuracy of three-dimensional (3D) MRI models created with this method. Shoulder MR images of 100 patients (average age, 44 years; range, 14-80 years; 60 men) were retrospectively collected from September 2013 to August 2018. CNNs were used to develop a fully automated segmentation model for proton density-weighted images. Shoulder MR images from an additional 50 patients (mean age, 33 years; range, 16-65 years; 35 men) were retrospectively collected from May 2014 to April 2019 to create 3D MRI glenohumeral models by transfer learning using Dixon-based sequences. Two musculoskeletal radiologists performed measurements on fully and semiautomated segmented 3D MRI models to assess glenohumeral anatomy, glenoid bone loss (GBL), and their impact on treatment selection. Performance of the CNNs was evaluated using Dice similarity coefficient (DSC), sensitivity, precision, and surface-based distance measurements. Measurements were compared using matched-pairs Wilcoxon signed rank test. The two-dimensional CNN model for the humerus and glenoid achieved a DSC of 0.95 and 0.86, a precision of 95.5% and 87.5%, an average precision of 98.6% and 92.3%, and a sensitivity of 94.8% and 86.1%, respectively. The 3D CNN model, for the humerus and glenoid, achieved a DSC of 0.95 and 0.86, precision of 95.1% and 87.1%, an average precision of 98.7% and 91.9%, and a sensitivity of 94.9% and 85.6%, respectively. There was no difference between glenoid and humeral head width fully and semiautomated 3D model measurements (P value range, .097-.99). CNNs could potentially be used in clinical practice to provide rapid and accurate 3D MRI glenohumeral bone models and GBL measurements. Supplemental material is available for this article. © RSNA, 2020.

Speech Emotion Recognition UsingConvolutional Neural Network and Long-Short TermMemory

Human-Robot interactions involve human intentions and human emotion. After the evolvement of positive psychology, the psychological research has a tremendous concentration to study the factors involved in the human emotion generation. Speech emotion recognition (SER) is a challenging job due to the complexity of emotions. Human emotion recognition is gaining importance as good emotional health can lead to good social and mental health. Although there are different approaches for speech emotion recognition, the most advanced model is Convolutional Neural Network (CNN) using Long Short-term Memory (LSTM) network. But they also suffer from the lack of parallelization of the sequences and computation times. Meanwhile, attention-mechanism has way better exhibitions in learning significant feature representations for specific tasks. Based on this technique, we propose an emotion recognition system with relation aware, self-attention mechanism to memorize the discriminative features for SER, where spectrograms are utilized as input. A CNN with a relation-aware self-attention is modelled to analyse 3D log-Mel spectrograms to extract the high-level features. Different layers such as 3D convolutional layers, 3D Max-pooling layers, and LSTM networks are used in the model. Here, the attention layer is exercised to support distinct parts of emotion and assemble discriminative utterance-level representations for SER. Finally, the fully connected layer is equipped with the utterance level representations with 64 output units to achieve higher-level representations. The approach of relation-aware attention-based 3D CNN and LSTM model provided a better outcome of 80.80% on average scale in speech emotion recognition. The proposed model in this paper focuses on enhancement of the attention mechanism to gain additional benefits of sequence to sequence parallelization by improving the recognition accuracy.

Brain MRI-based 3D Convolutional Neural Networks for Classification of Schizophrenia and Controls.

Convolutional Neural Network (CNN) has been successfully applied on classification of both natural images and medical images but limited studies applied it to differentiate patients with schizophrenia from healthy controls. Given the subtle, mixed, and sparsely distributed brain atrophy patterns of schizophrenia, the capability of automatic feature learning makes CNN a powerful tool for classifying schizophrenia from controls as it removes the subjectivity in selecting relevant spatial features. To examine the feasibility of applying CNN to classification of schizophrenia and controls based on structural Magnetic Resonance Imaging (MRI), we built 3D CNN models with different architectures and compared their performance with a handcrafted feature-based machine learning approach. Support vector machine (SVM) was used as classifier and Voxel-based Morphometry (VBM) was used as feature for handcrafted feature-based machine learning. 3D CNN models with sequential architecture, inception module and residual module were trained from scratch. CNN models achieved higher cross-validation accuracy than handcrafted feature-based machine learning. Moreover, testing on an independent dataset, 3D CNN models greatly outperformed handcrafted feature-based machine learning. This study underscored the potential of CNN for identifying patients with schizophrenia using 3D brain MR images and paved the way for imaging-based individual-level diagnosis and prognosis in psychiatric disorders.

Detection of Parkinson's Disease from 3T T1 Weighted MRI Scans Using 3D Convolutional Neural Network.

Parkinson’s Disease is a neurodegenerative disease that affects the aging population and is caused by a progressive loss of dopaminergic neurons in the substantia nigra pars compacta (SNc). With the onset of the disease, the patients suffer from mobility disorders such as tremors, bradykinesia, impairment of posture and balance, etc., and it progressively worsens in the due course of time. Additionally, as there is an exponential growth of the aging population in the world the number of people suffering from Parkinson’s Disease is increasing and it levies a huge economic burden on governments. However, until now no therapeutic method has been discovered for completely eradicating the disease from a person’s body after it’s onset. Therefore, the early detection of Parkinson’s Disease is of paramount importance to tackle the progressive loss of dopaminergic neurons in patients to serve them with a better life. In this study, 3T T1-weighted MRI scans were acquired from the Parkinson’s Progression Markers Initiative (PPMI) database of 406 subjects from baseline visit, where 203 were healthy and 203 were suffering from Parkinson’s Disease. Following data pre-processing, a 3D convolutional neural network (CNN) architecture was developed for learning the intricate patterns in the Magnetic Resonance Imaging (MRI) scans for the detection of Parkinson’s Disease. In the end, it was observed that the developed 3D CNN model performed superiorly by completely aligning with the hypothesis of the study and plotted an overall accuracy of 95.29%, average recall of 0.943, average precision of 0.927, average specificity of 0.9430, f1-score of 0.936, and Receiver Operating Characteristic—Area Under Curve (ROC-AUC) score of 0.98 for both the classes respectively.

Human Activity Recognition

The topic of Human activity recognition (HAR) is a prominent research area topic in the field of computer vision and image processing area. It has empowered state-of-art application in multiple sectors, surveillance, digital entertainment and medical healthcare. It is interesting to observe and intriguing to predict such kind of movements. Several sensor-based approaches have also been introduced to study and predict human activities such accelerometer, gyroscope, etc., it has its own advantages and disadvantages.[10] In this paper, an intelligent human activity recognition system is developed. Convolutional neural network (CNN) with spatiotemporal three dimensional (3D) kernels are trained using Kinetics data set which has 400 classes that depicts activities of humans in their everyday life and work and consist of 400 and more videos for each class. The 3D CNN model used in this model is RESNET-34. The videos were temporally cut down and last around tenth of a second. The trained model show satisfactory performance in all stages of training, testing. Finally the results show promising activity recognition of over 400 human actions.

Deep learning segmentation of gadolinium-enhancing lesions in multiple sclerosis

Objective: The aim of this study is to assess the performance of deep learning convolutional neural networks (CNNs) in segmenting gadolinium-enhancing lesions using a large cohort of multiple sclerosis (MS) patients. Methods: A three-dimensional (3D) CNN model was trained for segmentation of gadolinium-enhancing lesions using multispectral magnetic resonance imaging data (MRI) from 1006 relapsing–remitting MS patients. The network performance was evaluated for three combinations of multispectral MRI used as input: (U5) fluid-attenuated inversion recovery (FLAIR), T2-weighted, proton density-weighted, and pre- and post-contrast T1-weighted images; (U2) pre- and post-contrast T1-weighted images; and (U1) only post-contrast T1-weighted images. Segmentation performance was evaluated using the Dice similarity coefficient (DSC) and lesion-wise true-positive (TPR) and false-positive (FPR) rates. Performance was also evaluated as a function of enhancing lesion volume. Results: The DSC/TPR/FPR values averaged over all the enhancing lesion sizes were 0.77/0.90/0.23 using the U5 model. These values for the largest enhancement volumes (>500 mm3) were 0.81/0.97/0.04. For U2, the average DSC/TPR/FPR values were 0.72/0.86/0.31. Comparable performance was observed with U1. For all types of input, the network performance degraded with decreased enhancement size. Conclusion: Excellent segmentation of enhancing lesions was observed for enhancement volume ⩾70 mm3. The best performance was achieved when the input included all five multispectral image sets.

S152. DIAGNOSTIC CLASSIFICATION OF SCHIZOPHRENIA USING 3D CONVOLUTIONAL NEURAL NETWORK WITH RESTING-STATE FUNCTIONAL MRI

BackgroundSeveral machine-learning (ML) algorithms have been deployed in the diagnostic classification of schizophrenia. Compared to other ML methods, the 3D convolutional neural network (CNN) has an advantage of learning complex and subtle patterns in data and preserving spatial information, which is a more suitable tool for brain imaging data. Although resting-state functional MRI (rsfMRI) data has been used in previous ML studies relating to the diagnostic classification of schizophrenia, a limited number of studies have been conducted using resting-state functional connectivity resulted from group independent component analysis (ICA) and dual regression. The objective of this study was to investigate whether a successful diagnostic classification of schizophrenia vs. healthy controls could be achieved by the 3D CNN using resting-state networks in which areas with a significant group difference in activity existed.MethodsT1 and rsfMRI data were collected in 46 patients with recent-onset schizophrenia and 22 healthy controls. In the pre-processing steps of rsfMRI, the ICA-based automatic removal of motion artifacts was applied to subject-level ICA results and the resulting rsfMRI data were temporally concatenated for group ICA and dual regression. The executive control and auditory networks had areas with significantly higher activity in the control group compared with the patient group. The independent components (ICs) respective to the executive control and auditory networks were used as input for the 3D CNN model which was developed to discriminate the schizophrenia patients from the healthy controls.ResultsThe 3D CNN model using the executive control and auditory networks as inputs showed classification accuracies of 65~70%, and error rates of 30~35% approximately.DiscussionOur findings suggest that the 3D CNN model using rsfMRI data can be useful for learning patterns implicated in schizophrenia and identifying discriminative patterns of schizophrenia in brain imaging data.

PENet\u2014a scalable deep-learning model for automated diagnosis of pulmonary embolism using volumetric CT imaging

Pulmonary embolism (PE) is a life-threatening clinical problem and computed tomography pulmonary angiography (CTPA) is the gold standard for diagnosis. Prompt diagnosis and immediate treatment are critical to avoid high morbidity and mortality rates, yet PE remains among the diagnoses most frequently missed or delayed. In this study, we developed a deep learning model—PENet, to automatically detect PE on volumetric CTPA scans as an end-to-end solution for this purpose. The PENet is a 77-layer 3D convolutional neural network (CNN) pretrained on the Kinetics-600 dataset and fine-tuned on a retrospective CTPA dataset collected from a single academic institution. The PENet model performance was evaluated in detecting PE on data from two different institutions: one as a hold-out dataset from the same institution as the training data and a second collected from an external institution to evaluate model generalizability to an unrelated population dataset. PENet achieved an AUROC of 0.84 [0.82–0.87] on detecting PE on the hold out internal test set and 0.85 [0.81–0.88] on external dataset. PENet also outperformed current state-of-the-art 3D CNN models. The results represent successful application of an end-to-end 3D CNN model for the complex task of PE diagnosis without requiring computationally intensive and time consuming preprocessing and demonstrates sustained performance on data from an external institution. Our model could be applied as a triage tool to automatically identify clinically important PEs allowing for prioritization for diagnostic radiology interpretation and improved care pathways via more efficient diagnosis.

GraphDelta: MPNN Scoring Function for the Affinity Prediction of Protein-Ligand Complexes.

In this work, we present graph-convolutional neural networks for the prediction of binding constants of protein–ligand complexes. We derived the model using multi task learning, where the target variables are the dissociation constant (Kd), inhibition constant (Ki), and half maximal inhibitory concentration (IC50). Being rigorously trained on the PDBbind dataset, the model achieves the Pearson correlation coefficient of 0.87 and the RMSE value of 1.05 in pK units, outperforming recently developed 3D convolutional neural network model Kdeep.

Feature-shared adaptive-boost deep learning for invasiveness classification of pulmonary subsolid nodules in CT images.

In clinical practice, invasiveness is an important reference indicator for differentiating the malignant degree of subsolid pulmonary nodules. These nodules can be classified as atypical adenomatous hyperplasia (AAH), adenocarcinoma in situ (AIS), minimally invasive adenocarcinoma (MIA), or invasive adenocarcinoma (IAC). The automatic determination of a nodule's invasiveness based on chest CT scans can guide treatment planning. However, it is challenging, owing to the insufficiency of training data and their interclass similarity and intraclass variation. To address these challenges, we propose a two-stage deep learning strategy for this task: prior-feature learning followed by adaptive-boost deep learning. The adaptive-boost deep learning is proposed to train a strong classifier for invasiveness classification of subsolid nodules in chest CT images, using multiple 3D convolutional neural network (CNN)-based weak classifiers. Because ensembles of multiple deep 3D CNN models have a huge number of parameters and require large computing resources along with more training and testing time, the prior-feature learning is proposed to reduce the computations by sharing the CNN layers between all weak classifiers. Using this strategy, all weak classifiers can be integrated into a single network. Tenfold cross validation of binary classification was conducted on a total of 1357 nodules, including 765 noninvasive (AAH and AIS) and 592 invasive nodules (MIA and IAC). Ablation experimental results indicated that the proposed binary classifier achieved an accuracy of with an AUC of 81.3 . These results are superior compared to those achieved by three experienced chest imaging specialists who achieved an accuracy of , , and , respectively. About 200 additional nodules were also collected. These nodules covered 50 cases for each category (AAH, AIS, MIA, and IAC, respectively). Both binary and multiple classifications were performed on these data and the results demonstrated that the proposed method definitely achieves better performance than the performance achieved by nonensemble deep learning methods. It can be concluded that the proposed adaptive-boost deep learning can significantly improve the performance of invasiveness classification of pulmonary subsolid nodules in CT images, while the prior-feature learning can significantly reduce the total size of deep models. The promising results on clinical data show that the trained models can be used as an effective lung cancer screening tool in hospitals. Moreover, the proposed strategy can be easily extended to other similar classification tasks in 3D medical images.