3D Deep Learning Model Research Articles

Automatic hemorrhage segmentation on head CT scan for traumatic brain injury using 3D deep learning model

The most common cause of long-term disability and death in young adults is a traumatic brain injury. The decision for surgical intervention for craniotomy is dependent on the injury type and the patient's neurologic exam. The potential subtypes of intracranial hemorrhage that may necessitate surgical intervention include subdural hemorrhage, epidural hemorrhage, and intraparenchymal hemorrhage. We proposed a novel automatic method for segmenting the hemorrhage subtypes on a CT scan by integrated CT scan with bone window as input of a deep learning model. Brain CT scans were collected from adult patients and annotated regions of subdural hemorrhage, epidural hemorrhage, and intraparenchymal hemorrhage by neuroradiologists. Their raw DICOM images were preprocessed by two different window settings i.e., subdural and bone windows. The collected CT scans were divided into two datasets namely training and test datasets. A deep-learning model was modified to segment regions of each hemorrhage subtype. The model is a three-dimensional convolutional neural network including four parallel pathways that process the input at different resolutions. It was trained by a training dataset. After the segmentation result was produced by the deep-learning model, it was then improved in the post-processing step. The size of the segmented lesion was considered, and a region-growing algorithm was applied. We evaluated the performance of the proposed method on the test dataset. The method reached the median Dice similarity coefficients higher than 0.37 for each hemorrhage subtype. The proposed method demonstrates higher Dice similarity coefficients and improved segmentation performance compared to previously published literature.

Deep Neural Networks and Machine Learning Radiomics Modelling for Prediction of Relapse in Mantle Cell Lymphoma.

Simple SummaryMantle cell lymphoma (MCL) is an aggressive lymphoid tumour with a poor prognosis. There exist no routine biomarkers for the early prediction of relapse. Our study compared the potential of radiomics-based machine learning and 3D deep learning models as non-invasive biomarkers to risk-stratify MCL patients, thus promoting precision imaging in clinical oncology. Mantle cell lymphoma (MCL) is a rare lymphoid malignancy with a poor prognosis characterised by frequent relapse and short durations of treatment response. Most patients present with aggressive disease, but there exist indolent subtypes without the need for immediate intervention. The very heterogeneous behaviour of MCL is genetically characterised by the translocation t(11;14)(q13;q32), leading to Cyclin D1 overexpression with distinct clinical and biological characteristics and outcomes. There is still an unfulfilled need for precise MCL prognostication in real-time. Machine learning and deep learning neural networks are rapidly advancing technologies with promising results in numerous fields of application. This study develops and compares the performance of deep learning (DL) algorithms and radiomics-based machine learning (ML) models to predict MCL relapse on baseline CT scans. Five classification algorithms were used, including three deep learning models (3D SEResNet50, 3D DenseNet, and an optimised 3D CNN) and two machine learning models based on K-nearest Neighbor (KNN) and Random Forest (RF). The best performing method, our optimised 3D CNN, predicted MCL relapse with a 70% accuracy, better than the 3D SEResNet50 (62%) and the 3D DenseNet (59%). The second-best performing method was the KNN-based machine learning model (64%) after principal component analysis for improved accuracy. Our optimised CNN developed by ourselves correctly predicted MCL relapse in 70% of the patients on baseline CT imaging. Once prospectively tested in clinical trials with a larger sample size, our proposed 3D deep learning model could facilitate clinical management by precision imaging in MCL.

Combination of Radiological and Clinical Baseline Data for Outcome Prediction of Patients With an Acute Ischemic Stroke.

BackgroundAccurate prediction of clinical outcome is of utmost importance for choices regarding the endovascular treatment (EVT) of acute stroke. Recent studies on the prediction modeling for stroke focused mostly on clinical characteristics and radiological scores available at baseline. Radiological images are composed of millions of voxels, and a lot of information can be lost when representing this information by a single value. Therefore, in this study we aimed at developing prediction models that take into account the whole imaging data combined with clinical data available at baseline.MethodsWe included 3,279 patients from the MR CLEAN Registry; a prospective, observational, multicenter registry of patients with ischemic stroke treated with EVT. We developed two approaches to combine the imaging data with the clinical data. The first approach was based on radiomics features, extracted from 70 atlas regions combined with the clinical data to train machine learning models. For the second approach, we trained 3D deep learning models using the whole images and the clinical data. Models trained with the clinical data only were compared with models trained with the combination of clinical and image data. Finally, we explored feature importance plots for the best models and identified many known variables and image features/brain regions that were relevant in the model decision process.ResultsFrom 3,279 patients included, 1,241 (37%) patients had a good functional outcome [modified Rankin Scale (mRS) ≤ 2] and 1,954 (60%) patients had good reperfusion [modified Thrombolysis in Cerebral Infarction (eTICI) ≥ 2b]. There was no significant improvement by combining the image data to the clinical data for mRS prediction [mean area under the receiver operating characteristic (ROC) curve (AUC) of 0.81 vs. 0.80] above using the clinical data only, regardless of the approach used. Regarding predicting reperfusion, there was a significant improvement when image and clinical features were combined (mean AUC of 0.54 vs. 0.61), with the highest AUC obtained by the deep learning approach.ConclusionsThe combination of radiomics and deep learning image features with clinical data significantly improved the prediction of good reperfusion. The visualization of prediction feature importance showed both known and novel clinical and imaging features with predictive values.

Fully Automatic Classification of Brain Atrophy on NCCT Images in Cerebral Small Vessel Disease: A Pilot Study Using Deep Learning Models.

ObjectiveBrain atrophy is an important imaging characteristic of cerebral small vascular disease (CSVD). Our study explores the linear measurement application on CT images of CSVD patients and develops a fully automatic brain atrophy classification model. The second aim was to compare it with the end-to-end Convolutional Neural Networks (CNNs) model.MethodsA total of 385 subjects such as 107 no-atrophy brain, 185 mild atrophy, and 93 severe atrophy were collected and randomly separated into training set (n = 308) and test set (n = 77). Key slices for linear measurement were manually identified and used to annotate nine linear measurements and a binary classification of cerebral sulci widening. A linear-measurement-based pipeline (2D model) was constructed for two-types (existence/non-existence brain atrophy) or three-types classification (no/mild atrophy/severe atrophy). For comparison, an end-to-end CNN model (3D-deep learning model) for brain atrophy classification was also developed. Furthermore, age and gender were integrated to the 2D and 3D models. The sensitivity, specificity, accuracy, average F1 score, receiver operating characteristics (ROC) curves for two-type classification and weighed kappa for three-type classification of the two models were compared.ResultsAutomated measurement of linear measurements and cerebral sulci widening achieved moderate to almost perfect agreement with manual annotation. In two-type atrophy classification, area under the curves (AUCs) of the 2D model and 3D model were 0.953 and 0.941 with no significant difference (p = 0.250). The Weighted kappa of the 2D model and 3D model were 0.727 and 0.607 according to standard classification they displayed, mild atrophy and severe atrophy, respectively. Applying patient age and gender information improved classification performances of both 2D and 3D models in two-type and three-type classification of brain atrophy.ConclusionWe provide a model composed of different modules that can classify CSVD-related brain atrophy on CT images automatically, using linear measurement. It has similar performance and better interpretability than the end-to-end CNNs model and may prove advantageous in the clinical setting.

MultiR-Net: A Novel Joint Learning Network for COVID-19 segmentation and classification

The outbreak of COVID-19 has caused a severe shortage of healthcare resources. Ground Glass Opacity (GGO) and consolidation of chest CT scans have been an essential basis for imaging diagnosis since 2020. The similarity of imaging features between COVID-19 and other pneumonia makes it challenging to distinguish between them and affects radiologists' diagnosis. Recently, deep learning in COVID-19 has been mainly divided into disease classification and lesion segmentation, yet little work has focused on the feature correlation between the two tasks. To address these issues, in this study, we propose MultiR-Net, a 3D deep learning model for combined COVID-19 classification and lesion segmentation, to achieve real-time and interpretable COVID-19 chest CT diagnosis. Precisely, the proposed network consists of two subnets: a multi-scale feature fusion UNet-like subnet for lesion segmentation and a classification subnet for disease diagnosis. The features between the two subnets are fused by the reverse attention mechanism and the iterable training strategy. Meanwhile, we proposed a loss function to enhance the interaction between the two subnets. Individual metrics can not wholly reflect network effectiveness. Thus we quantify the segmentation results with various evaluation metrics such as average surface distance, volume Dice, and test on the dataset. We employ a dataset containing 275 3D CT scans for classifying COVID-19, Community-acquired Pneumonia (CAP), and healthy people and segmented lesions in pneumonia patients. We split the dataset into 70% and 30% for training and testing. Extensive experiments showed that our multi-task model framework obtained an average recall of 93.323%, an average precision of 94.005% on the classification test set, and a 69.95% Volume Dice score on the segmentation test set of our dataset.

Locoregional Recurrence Prediction Using a Deep Neural Network of Radiological and Radiotherapy Images.

Radiation therapy (RT) is an important and potentially curative modality for head and neck squamous cell carcinoma (HNSCC). Locoregional recurrence (LR) of HNSCC after RT is ranging from 15% to 50% depending on the primary site and stage. In addition, the 5-year survival rate of patients with LR is low. To classify high-risk patients who might develop LR, a deep learning model for predicting LR needs to be established. In this work, 157 patients with HNSCC who underwent RT were analyzed. Based on the National Cancer Institute’s multi-institutional TCIA data set containing FDG-PET/CT/dose, a 3D deep learning model was proposed to predict LR without time-consuming segmentation or feature extraction. Our model achieved an averaged area under the curve (AUC) of 0.856. Adding clinical factors into the model improved the AUC to an average of 0.892 with the highest AUC of up to 0.974. The 3D deep learning model could perform individualized risk quantification of LR in patients with HNSCC without time-consuming tumor segmentation.

Advanced Warning of Aortic Dissection on Non-Contrast CT: The Combination of Deep Learning and Morphological Characteristics.

Background: The identification of aortic dissection (AD) at baseline plays a crucial role in clinical practice. Non-contrast CT scans are widely available, convenient, and easy to perform. However, the detection of AD on non-contrast CT scans by radiologists currently lacks sensitivity and is suboptimal.Methods: A total of 452 patients who underwent aortic CT angiography (CTA) were enrolled retrospectively from two medical centers in China to form the internal cohort (341 patients, 139 patients with AD, 202 patients with non-AD) and the external testing cohort (111 patients, 46 patients with AD, 65 patients with non-AD). The internal cohort was divided into the training cohort (n = 238), validation cohort (n = 35), and internal testing cohort (n = 68). Morphological characteristics were extracted from the aortic segmentation. A deep-integrated model based on the Gaussian Naive Bayes algorithm was built to differentiate AD from non-AD, using the combination of the three-dimensional (3D) deep-learning model score and morphological characteristics. The areas under the receiver operating characteristic curve (AUCs), accuracy, sensitivity, and specificity were used to evaluate the model performance. The proposed model was also compared with the subjective assessment of radiologists.Results: After the combination of all the morphological characteristics, our proposed deep-integrated model significantly outperformed the 3D deep-learning model (AUC: 0.948 vs. 0.803 in the internal testing cohort and 0.969 vs. 0.814 in the external testing cohort, both p < 0.05). The accuracy, sensitivity, and specificity of our model reached 0.897, 0.862, and 0.923 in the internal testing cohort and 0.730, 0.978, and 0.554 in the external testing cohort, respectively. The accuracy for AD detection showed no significant difference between our model and the radiologists (p > 0.05).Conclusion: The proposed model presented good performance for AD detection on non-contrast CT scans; thus, early diagnosis and prompt treatment would be available.

Fusing 2D and 3D convolutional neural networks for the segmentation of aorta and coronary arteries from CT images

Automated segmentation of three-dimensional medical images is of great importance for the detection and quantification of certain diseases such as stenosis in the coronary arteries. Many 2D and 3D deep learning models, especially deep convolutional neural networks (CNNs), have achieved state-of-the-art segmentation performance on 3D medical images. Yet, there is a trade-off between the field of view and the utilization of inter-slice information when using pure 2D or 3D CNNs for 3D segmentation, which compromises the segmentation accuracy. In this paper, we propose a two-stage strategy that retains the advantages of both 2D and 3D CNNs and apply the method for the segmentation of the human aorta and coronary arteries, with stenosis, from computed tomography (CT) images. In the first stage, a 2D CNN, which can extract large-field-of-view information, is used to segment the aorta and coronary arteries simultaneously in a slice-by-slice fashion. Then, in the second stage, a 3D CNN is applied to extract the inter-slice information to refine the segmentation of the coronary arteries in certain subregions not resolved well in the first stage. We show that the 3D network of the second stage can improve the continuity between slices and reduce the missed detection rate of the 2D CNN. Compared with directly using a 3D CNN, the two-stage approach can alleviate the class imbalance problem caused by the large non-coronary artery (aorta and background) and the small coronary artery and reduce the training time because the vast majority of negative voxels are excluded in the first stage. To validate the efficacy of our method, extensive experiments are carried out to compare with other approaches based on pure 2D or 3D CNNs and those based on hybrid 2D-3D CNNs.

Deep learning methods for automatic segmentation of lower leg muscles and bones from MRI scans of children with and without cerebral palsy.

Cerebral palsy is a neurological condition that is known to affect muscle growth. Detailed investigations of muscle growth require segmentation of muscles from MRI scans, which is typically done manually. In this study, we evaluated the performance of 2D, 3D, and hybrid deep learning models for automatic segmentation of 11 lower leg muscles and two bones from MRI scans of children with and without cerebral palsy. All six models were trained and evaluated on manually segmented T1 -weighted MRI scans of the lower legs of 20 children, six of whom had cerebral palsy. The segmentation results were assessed using the median Dice similarity coefficient (DSC), average symmetric surface distance (ASSD), and volume error (VError) of all 13 labels of every scan. The best performance was achieved by H-DenseUNet, a hybrid model (DSC 0.90, ASSD 0.5 mm, and VError 2.6 cm3 ). The performance was equivalent to the inter-rater performance of manual segmentation (DSC 0.89, ASSD 0.6 mm, and VError 3.3 cm3 ). Models trained with the Dice loss function outperformed models trained with the cross-entropy loss function. Near-optimal performance could be attained using only 11 scans for training. Segmentation performance was similar for scans of typically developing children (DSC 0.90, ASSD 0.5 mm, and VError 2.8 cm3 ) and children with cerebral palsy (DSC 0.85, ASSD 0.6 mm, and VError 2.4 cm3 ). These findings demonstrate the feasibility of fully automatic segmentation of individual muscles and bones from MRI scans of children with and without cerebral palsy.

Application of Hyperspectral Imaging for Maturity and Soluble Solids Content Determination of Strawberry With Deep Learning Approaches.

Maturity degree and quality evaluation are important for strawberry harvest, trade, and consumption. Deep learning has been an efficient artificial intelligence tool for food and agro-products. Hyperspectral imaging coupled with deep learning was applied to determine the maturity degree and soluble solids content (SSC) of strawberries with four maturity degrees. Hyperspectral image of each strawberry was obtained and preprocessed, and the spectra were extracted from the images. One-dimension residual neural network (1D ResNet) and three-dimension (3D) ResNet were built using 1D spectra and 3D hyperspectral image as inputs for maturity degree evaluation. Good performances were obtained for maturity identification, with the classification accuracy over 84% for both 1D ResNet and 3D ResNet. The corresponding saliency maps showed that the pigments related wavelengths and image regions contributed more to the maturity identification. For SSC determination, 1D ResNet model was also built, with the determination of coefficient (R2) over 0.55 of the training, validation, and testing sets. The saliency maps of 1D ResNet for the SSC determination were also explored. The overall results showed that deep learning could be used to identify strawberry maturity degree and determine SSC. More efforts were needed to explore the use of 3D deep learning methods for the SSC determination. The close results of 1D ResNet and 3D ResNet for classification indicated that more samples might be used to improve the performances of 3D ResNet. The results in this study would help to develop 1D and 3D deep learning models for fruit quality inspection and other researches using hyperspectral imaging, providing efficient analysis approaches of fruit quality inspection using hyperspectral imaging.

FULL AUTOMATION OF EMBRYO EVALUATION MODELS BENEFITS FROM TRAINING ON BOTH TRANSFERRED AND DISCARDED EMBRYOS

Is overall performance of an AI-based embryo evaluation model improved by including discarded embryos during training compared to using only transferred embryos with known implantation data (KID)? A data set of 14,643 KID embryos with known fetal heart (4,337 FH+ and 10,307 FH-) incubated for at least 4 days was obtained from 18 different clinics. The data set also included 23,674 embryos either cryopreserved or transferred with unknown FH and 101,153 embryos that were discarded and thus not cryopreserved or transferred. Two 3D convolutional deep learning models were trained on 85% of the data set. The first model (KID-o) was trained only on KID embryos. The second model (KID-d) was trained on both KID embryos and discarded embryos, with discarded embryos pseudo-labelled as FH-. The KID-d model training used oversampling of FH+ embryos to ensure equal distribution of FH+ and FH- during training. The models were evaluated with an internal validation data set based on the remaining 15% of the data. In addition, an external validation data set with 1,125 KID embryos, 6,327 embryos either cryopreserved or transferred with unknown implantation data, and 9,728 discarded embryos was obtained from American Hospital, Turkey. The main goal of embryo evaluation is to rank embryos to determine the order of transfer. This task was evaluated by calculating the area under the curve (AUC) for KID embryos. Evaluation was based solely on clinical outcome. A second goal is to categorize embryos into useable (i.e. transferred or cryopreserved) or discarded embryos. This task was evaluated by finding the threshold where at least 95% of the useable embryos were classified as useable. At this 95% sensitivity threshold, the specificity was calculated. Thus, this evaluation only concerned if embryos were used or discarded and was not related to clinical outcome. The ranking AUCs of KID embryos by the two models were nearly identical. Thus, the AUCs for the KID-o and KID-d model, respectively, were 0.667 and 0.670 for the internal validation, and 0.715 and 0.714 for the external validation. However, the specificities for categorization of useable embryos were significantly (p<0.001 – McNemar’s test) higher for the KID-d than the KID-o model. Thus, the specificities for the KID-o and KID-d model, respectively, were 0.48 and 0.82 for the internal validation, and 0.50 and 0.84 for the external validation. The results showed that the performance for ranking pre-selected transferred KID embryos was not negatively affected by including discarded embryos in the training. However, for categorizing embryos into useable or discarded embryos, the model performed significantly better with the inclusion of discarded embryos. Thus, overall, the best performance was observed if discarded embryos were included in the training.

A 3D deep learning model to predict the diagnosis of dementia with Lewy bodies, Alzheimer\u2019s disease, and mild cognitive impairment using brain 18F-FDG PET

PurposeThe purpose of this study is to develop and validate a 3D deep learning model that predicts the final clinical diagnosis of Alzheimer’s disease (AD), dementia with Lewy bodies (DLB), mild cognitive impairment due to Alzheimer’s disease (MCI-AD), and cognitively normal (CN) using fluorine 18 fluorodeoxyglucose PET (18F-FDG PET) and compare model’s performance to that of multiple expert nuclear medicine physicians’ readers.Materials and methodsRetrospective 18F-FDG PET scans for AD, MCI-AD, and CN were collected from Alzheimer’s disease neuroimaging initiative (556 patients from 2005 to 2020), and CN and DLB cases were from European DLB Consortium (201 patients from 2005 to 2018). The introduced 3D convolutional neural network was trained using 90% of the data and externally tested using 10% as well as comparison to human readers on the same independent test set. The model’s performance was analyzed with sensitivity, specificity, precision, F1 score, receiver operating characteristic (ROC). The regional metabolic changes driving classification were visualized using uniform manifold approximation and projection (UMAP) and network attention.ResultsThe proposed model achieved area under the ROC curve of 96.2% (95% confidence interval: 90.6–100) on predicting the final diagnosis of DLB in the independent test set, 96.4% (92.7–100) in AD, 71.4% (51.6–91.2) in MCI-AD, and 94.7% (90–99.5) in CN, which in ROC space outperformed human readers performance. The network attention depicted the posterior cingulate cortex is important for each neurodegenerative disease, and the UMAP visualization of the extracted features by the proposed model demonstrates the reality of development of the given disorders.ConclusionUsing only 18F-FDG PET of the brain, a 3D deep learning model could predict the final diagnosis of the most common neurodegenerative disorders which achieved a competitive performance compared to the human readers as well as their consensus.

3D Deep Learning Model for the Pretreatment Evaluation of Treatment Response in Esophageal Carcinoma: A Prospective Study (ChiCTR2000039279)

To develop and validate a pretreatment computed tomography (CT)-based deep-learning (DL) model for predicting the treatment response to concurrent chemoradiation therapy (CCRT) among patients with locally advanced thoracic esophageal squamous cell carcinoma (TESCC). We conducted a prospective, multicenter study on the therapeutic efficacy of CCRT among TESCC patients across 9 hospitals in China (ChiCTR2000039279). A total of 306 patients with locally advanced TESCC diagnosed by histopathology from August 2015 to May 2020 were included in this study. A 3-dimensional DL radiomics model (3D-DLRM) was developed and validated based on pretreatment CT images to predict the response to CCRT. Furthermore, the prediction performance of the newly developed 3D-DLRM was analyzed according to 3 categories: radiation therapy plan, radiation field, and prescription dose used. The 3D-DLRM achieved good prediction performance, with areas under the receiver operating characteristic curve of 0.897 (95% confidence interval, 0.840-0.959) for the training cohort and 0.833 (95% confidence interval, 0.654-1.000) for the validation cohort. Specifically, the 3D-DLRM accurately predicted patients who would not respond to CCRT, with a positive predictive value (PPV) of 100% for the validation cohort. Moreover, the 3D-DLRM performed well in all 3 categories, each with areas under the receiver operating characteristic curve of >0.8 and positive predictive values of approximately 100%. The proposed pretreatment CT-based 3D-DLRM provides a potential tool for predicting the response to CCRT among patients with locally advanced TESCC. With the help of precise pretreatment prediction, we may guide the individualized treatment of patients and improve survival.

Diagnosis of obstructive sleep apnea with prediction of flow characteristics according to airway morphology automatically extracted from medical images: Computational fluid dynamics and artificial intelligence approach

BackgroundObstructive sleep apnea syndrome (OSAS) is being observed in an increasing number of cases. It can be diagnosed using several methods such as polysomnography. ObjectivesTo overcome the challenges of time and cost faced by conventional diagnostic methods, this paper proposes computational fluid dynamics (CFD) and machine-learning approaches that are derived from the upper-airway morphology with automatic segmentation using deep learning. MethodWe adopted a 3D UNet deep-learning model to perform medical image segmentation. 3D UNet prevents the feature-extraction loss that may occur by concatenating layers and extracts the anteroposterior coordination and width of the airway morphology. To create flow characteristics of the upper airway training data, we analyzed the changes in flow characteristics according to the upper-airway morphology using CFD. A multivariate Gaussian process regression (MVGPR) model was used to train the flow characteristic values. The trained MVGPR enables the prompt prediction of the aerodynamic features of the upper airway without simulation. Unlike conventional regression methods, MVGPR can be trained by considering the correlation between the flow characteristics. As a diagnostic step, a support vector machine (SVM) with predicted aerodynamic and biometric features was used in this study to classify patients as healthy or suffering from moderate OSAS. SVM is beneficial as it is easy to learn even with a small dataset, and it can diagnose various flow characteristics as factors while enhancing the feature via the kernel function. As the patient dataset is small, the Monte Carlo cross-validation was used to validate the trained model. Furthermore, to overcome the imbalanced data problem, the oversampling method was applied. ResultThe segmented upper-airway results of the high-resolution and low-resolution models present overall average dice coefficients of 0.76±0.041 and 0.74±0.052, respectively. Furthermore, the classification accuracy, sensitivity, specificity, and F1-score of the diagnosis algorithm were 81.5%, 89.3%, 86.2%, and 87.6%, respectively. ConclusionThe convenience and accuracy of sleep apnea diagnosis are improved using deep learning and machine learning. Further, the proposed method can aid clinicians in making appropriate decisions to evaluate the possible applications of OSAS.

ACEnet: Anatomical context-encoding network for neuroanatomy segmentation.

Segmentation of brain structures from magnetic resonance (MR) scans plays an important role in the quantification of brain morphology. Since 3D deep learning models suffer from high computational cost, 2D deep learning methods are favored for their computational efficiency. However, existing 2D deep learning methods are not equipped to effectively capture 3D spatial contextual information that is needed to achieve accurate brain structure segmentation. In order to overcome this limitation, we develop an Anatomical Context-Encoding Network (ACEnet) to incorporate 3D spatial and anatomical contexts in 2D convolutional neural networks (CNNs) for efficient and accurate segmentation of brain structures from MR scans, consisting of 1) an anatomical context encoding module to incorporate anatomical information in 2D CNNs and 2) a spatial context encoding module to integrate 3D image information in 2D CNNs. In addition, a skull stripping module is adopted to guide the 2D CNNs to attend to the brain. Extensive experiments on three benchmark datasets have demonstrated that our method achieves promising performance compared with state-of-the-art alternative methods for brain structure segmentation in terms of both computational efficiency and segmentation accuracy.

A 3D Deep Learning Model for Rapid Prediction of Structural Dynamics of Workpieces During Machining

Chatter stability in machining flexible parts depends directly on the structural dynamics of the workpiece. This paper proposes a novel two-stage framework that combines finite element (FE) and data-driven deep learning techniques to rapidly predict the varying dynamics of workpieces during machining. An automated framework is developed to create a large training dataset of CAD models with gradually-changing geometries. A deep 3D Convolutional Neural Network (3D-CNN) is developed to “learn” the variations in dynamic parameters as a function of geometry. The current model has been successfully implemented for prediction of natural frequencies of workpieces during turning operations. The proposed framework can be used as a computationally efficient tool in online process monitoring and automated correction applications.

Multi‐view frontal face image generation: A survey

AbstractFace images from different perspectives reduce the accuracy of face recognition, and the generation of frontal face images is an important research topic in the field of face recognition. To understand the development of frontal face generation models and grasp the current research hotspots and trends, existing methods based on 3D models, deep learning, and hybrid models are summarized, and the current commonly used face generation methods are introduced. Dataset, and compare the performance of existing models through experiments. The purpose of this paper is to fundamentally understand the advantages of existing frontal face generation, sort out the key issues of such generation, and look toward future development trends.

Attention-Guided 3D-CNN Framework for Glaucoma Detection and Structural-Functional Association Using Volumetric Images.

The direct analysis of 3D Optical Coherence Tomography (OCT) volumes enables deep learning models (DL) to learn spatial structural information and discover new bio-markers that are relevant to glaucoma. Downsampling 3D input volumes is the state-of-art solution to accommodate for the limited number of training volumes as well as the available computing resources. However, this limits the network's ability to learn from small retinal structures in OCT volumes. In this paper, our goal is to improve the performance by providing guidance to DL model during training in order to learn from finer ocular structures in 3D OCT volumes. Therefore, we propose an end-to-end attention guided 3D DL model for glaucoma detection and estimating visual function from retinal structures. The model consists of three pathways with the same network architecture but different inputs. One input is the original 3D-OCT cube and the other two are computed during training guided by the 3D gradient class activation heatmaps. Each pathway outputs the class-label and the whole model is trained concurrently to minimize the sum of losses from three pathways. The final output is obtained by fusing the predictions of the three pathways. Also, to explore the robustness and generalizability of the proposed model, we apply the model on a classification task for glaucoma detection as well as a regression task to estimate visual field index (VFI) (a value between 0 and 100). A 5-fold cross-validation with a total of 3782 and 10,370 OCT scans is used to train and evaluate the classification and regression models, respectively. The glaucoma detection model achieved an area under the curve (AUC) of 93.8% compared with 86.8% for a baseline model without the attention-guided component. The model also outperformed six different feature based machine learning approaches that use scanner computed measurements for training. Further, we also assessed the contribution of different retinal layers that are relevant to glaucoma. The VFI estimation model achieved a Pearson correlation and median absolute error of 0.75 and 3.6%, respectively, for a test set of size 3100 cubes.

Segmentation of unbalanced and in-homogeneous point clouds and its application to 3D scanned trees

Segmentation of 3D point clouds is still an open issue in the case of unbalanced and in-homogeneous data-sets. In the application context of the modeling of botanical trees, a fundamental challenge consists in separating the leaves from the wood. Based on deep learning and a class decision process, we propose an innovative method designed to separate leaf points from wood points in terrestrial LiDAR point clouds of trees. Although simple, our approach learns trees characteristic point patterns efficiently and robustly. To train our 3D deep learning model, we constructed a 3D labeled point cloud data-set of different tree species. Experiments show that our 3D deep representation together with our geometric approach leads to significant improvement over the state-of-the-art methods in segmentation task.

Development and Validation of a Modified Three-Dimensional U-Net Deep-Learning Model for Automated Detection of Lung Nodules on Chest CT Images From the Lung Image Database Consortium and Japanese Datasets

A more accurate lung nodule detection algorithm is needed. We developed a modified three-dimensional (3D) U-net deep-learning model for the automated detection of lung nodules on chest CT images. The purpose of this study was to evaluate the accuracy of the developed modified 3D U-net deep-learning model. In this Health Insurance Portability and Accountability Act-compliant, Institutional Review Board-approved retrospective study, the 3D U-net based deep-learning model was trained using the Lung Image Database Consortium and Image Database Resource Initiative dataset. For internal model validation, we used 89 chest CT scans that were not used for model training. For external model validation, we used 450 chest CT scans taken at an urban university hospital in Japan. Each case included at least one nodule of >5 mm identified by an experienced radiologist. We evaluated model accuracy using the competition performance metric (CPM) (average sensitivity at 1/8, 1/4, 1/2, 1, 2, 4, and 8 false-positives per scan). The 95% confidence interval (CI) was computed by bootstrapping 1000 times. In the internal validation, the CPM was 94.7% (95% CI: 89.1%-98.6%). In the external validation, the CPM was 83.3% (95% CI: 79.4%-86.1%). The modified 3D U-net deep-learning model showed high performance in both internal and external validation.