Deep Learning-based Diagnosis Research Articles

Deep learning-based diagnosis and survival prediction of patients with renal cell carcinoma from primary whole slide images

There is an urgent clinical demand to explore novel diagnostic and prognostic biomarkers for renal cell carcinoma (RCC). We proposed deep learning-based artificial intelligence strategies. The study included 1752 whole slide images from multiple centres. Based on the pixel-level of RCC segmentation, the diagnosis diagnostic model achieved an area under the receiver operating characteristic curve (AUC) of 0.977 (95% CI 0.969-0.984) in the external validation cohort. In addition, our diagnostic model exhibited excellent performance in the differential diagnosis of RCC from renal oncocytoma, which achieved an AUC of 0.951 (0.922-0.972). The graderisk for the recognition of high-grade tumour achieved AUCs of 0.840 (0.805-0.871) in the Cancer Genome Atlas (TCGA) cohort, 0.857 (0.813-0.894) in the Shanghai General Hospital (General) cohort, and 0.894 (0.842-0.933) in the Clinical Proteomic Tumor Analysis Consortium (CPTAC) cohort, for the recognition of high-grade tumour. The OSrisk for predicting 5-year survival status achieved an AUC of 0.784 (0.746-0.819) in the TCGA cohort, which was further verified in the independent general cohort and the CPTAC cohort, with AUCs of 0.774 (0.723-0.820) and 0.702 (0.632-0.765), respectively. Moreover, the competing-risk nomogram (CRN) showed its potential to be a prognostic indicator, with a hazard ratio (HR) of 5.664 (3.893-8.239, p<0.0001), outperforming other traditional clinical prognostic indicators. Kaplan-Meier survival analysis further illustrated that our CRN could significantly distinguish patients with high survival risk. Deep learning-based artificial intelligence could be a useful tool for clinicians to diagnose and predict the prognosis of RCC patients, thus improving the process of individualised treatment.

Removing Adversarial Noise in X-ray Images via Total Variation Minimizationand Patch-Based Regularization for Robust DeepLearning-based Diagnosis.

Deep learning has significantly advanced the field of radiology-based disease diagnosis, offering enhanced accuracy and efficiency in detecting various medical conditions through the analysis of complex medical images such as X-rays. This technology's ability to discern subtle patterns and anomalies has proven invaluable for swift and accurate disease identification. The relevance of deep learning in radiology has been particularly highlighted during the COVID-19 pandemic, where rapid and accurate diagnosis is crucial for effective treatment and containment. However, recent research has uncovered vulnerabilities in deep learning models when exposed to adversarial attacks, leading to incorrect predictions. In response to this critical challenge, we introduce a novel approach that leverages total variation minimization to combat adversarial noise within X-ray images effectively. Our focus narrows to COVID-19 diagnosis as a case study, where we initially construct a classification model through transfer learning designed to accurately classify lung X-ray images encompassing no pneumonia, COVID-19 pneumonia, and non-COVID pneumonia cases. Subsequently, we extensively evaluated the model's susceptibility to targeted and un-targetedadversarial attacksby employing the fast gradient sign gradient (FGSM) method. Our findings reveal a substantial reduction in the model's performance, with the average accuracy plummeting from 95.56 to 19.83% under adversarial conditions. However, the experimental results demonstrate the exceptional efficacy of the proposed denoising approach in enhancing the performance of diagnosis models when applied to adversarial examples. Post-denoising, the model exhibits a remarkable accuracy improvement, surging from 19.83 to 88.23% on adversarial images. These promising outcomes underscore the potential of denoising techniques to fortify the resilience and reliability of AI-based COVID-19 diagnostic systems, laying the foundation for their successful deployment in clinical settings.

A deep learning-based automated diagnosis system for SPECT myocardial perfusion imaging

Images obtained from single-photon emission computed tomography for myocardial perfusion imaging (MPI SPECT) contain noises and artifacts, making cardiovascular disease diagnosis difficult. We developed a deep learning-based diagnosis support system using MPI SPECT images. Single-center datasets of MPI SPECT images (n = 5443) were obtained and labeled as healthy or coronary artery disease based on diagnosis reports. Three axes of four-dimensional datasets, resting, and stress conditions of three-dimensional reconstruction data, were reconstructed, and an AI model was trained to classify them. The trained convolutional neural network showed high performance [area under the curve (AUC) of the ROC curve: approximately 0.91; area under the recall precision curve: 0.87]. Additionally, using unsupervised learning and the Grad-CAM method, diseased lesions were successfully visualized. The AI-based automated diagnosis system had the highest performance (88%), followed by cardiologists with AI-guided diagnosis (80%) and cardiologists alone (65%). Furthermore, diagnosis time was shorter for AI-guided diagnosis (12 min) than for cardiologists alone (31 min). Our high-quality deep learning-based diagnosis support system may benefit cardiologists by improving diagnostic accuracy and reducing working hours.

Multibranch CNN With MLP-Mixer-Based Feature Exploration for High-Performance Disease Diagnosis.

Deep learning-based diagnosis is becoming an indispensable part of modern healthcare. For high-performance diagnosis, the optimal design of deep neural networks (DNNs) is a prerequisite. Despite its success in image analysis, existing supervised DNNs based on convolutional layers often suffer from their rudimentary feature exploration ability caused by the limited receptive field and biased feature extraction of conventional convolutional neural networks (CNNs), which compromises the network performance. Here, we propose a novel feature exploration network named manifold embedded multilayer perceptron (MLP) mixer (ME-Mixer), which utilizes both supervised and unsupervised features for disease diagnosis. In the proposed approach, a manifold embedding network is employed to extract class-discriminative features; then, two MLP-Mixer-based feature projectors are adopted to encode the extracted features with the global reception field. Our ME-Mixer network is quite general and can be added as a plugin to any existing CNN. Comprehensive evaluations on two medical datasets are performed. The results demonstrate that their approach greatly enhances the classification accuracy in comparison with different configurations of DNNs with acceptable computational complexity.

Explainable tele-ultrasound diagnosis of blast lung injury via incremental multimodal multistage fusion and video transformer

Blast lung injury (BLI) is a significant concern in cases of explosive events and can have severe consequences for victims. The injury is often irreversible, emphasizing the need for early symptom confirmation and intervention. Tele-ultrasound (Tele-US) is a novel approach that enables healthcare professionals to remotely interpret US images and provide diagnostic assessments, which can offer timely and accurate assessments for BLI. In this study, we propose an explainable two-stream tele-US diagnosis method for BLI, which is an incremental multimodal multistage fusion method (IM3S-Fusion). This method is the first deep learning-based diagnosis method for BLI and, specifically, the first exploration of video Transformers in this field. It integrates optical flow into US videos, enhancing model interpretability and improving diagnostic accuracy. By integrating incremental broad learning system (IBLS), IM3S-Fusion can incrementally update its parameters with new samples in a lifelong learning fashion, especially for those misdiagnosed samples. The results show its superiority in diagnosing BLI with (87.50 ± 2.76)%, (89.06 ± 2.71)%, (88.09 ± 5.26)%, (85.71 ± 7.14)% and (88.43 ± 2.24)% in accuracy, recall, precision, specificity and F1-score, respectively. Explanatory assistance including fused US video with optical flow, class activation map videos and confidence scores, is provided for doctors to facilitate their diagnosis.

Deep learning-based diagnosis of aortic dissection using an electrocardiogram: Development, validation, and clinical implications of the AADE score.

Aortic dissection (AD) is frequently associated with abnormalities in electrocardiographic findings. Advancements in medical technology present an opportunity to leverage these observations to improve patient diagnosis and care. This study aimed to develop a deep learning artificial intelligence (AI) model for AD detection using electrocardiograms (ECGs) and introduce the AI-Aortic-Dissection-ECG (AADE) score to provide clinicians with a measure to determine AD severity. From a cohort of 1878 patients, including 313 with AD, and 313 with chest pain (control group), we created training and validation subsets (7:3 ratio). A convolutional neural networks (CNN) model was trained for AD detection, with performance metrics like accuracy and F1 score (the harmonic mean of precision and recall) monitored. The AI-derived AADE score (0-1) was investigated against clinical parameters and ECG features over a median follow-up of 21.2 months. The CNN model demonstrated robust performance with an accuracy of 0.93 and an F1 score of 0.93 for the AD group, and an accuracy of 0.871 with an F1 score of 0.867 for the chest pain group. The AADE score showed correlations with specific ECG patterns and demonstrated that higher scores aligned with increased mortality risk. Our CNN-based AI model offers a promising approach for AD detection using ECG. The AADE score, based on AI, can serve as a pivotal tool in refining clinical assessments and management strategies.

Melanoma Breslow Thickness Classification using Ensemble-based Knowledge Distillation with Semi-supervised Convolutional Neural Networks.

Melanoma is considered a global public health challenge and is responsible for more than 90% deaths related to skin cancer. Although the diagnosis of early melanoma is the main goal of dermoscopy, the discrimination between dermoscopic images of in situ and invasive melanomas can be a difficult task even for experienced dermatologists. Recent advances in artificial intelligence in the field of medical image analysis show that its application to dermoscopy with the aim of supporting and providing a second opinion to the medical expert could be of great interest. In this work, four datasets from different sources were used to train and evaluate deep learning models on in situ versus invasive melanoma classification and on Breslow thickness prediction. Supervised learning and semi-supervised learning using a multi-teacher ensemble knowledge distillation approach were considered and evaluated using a stratified 5-fold cross-validation scheme. The best models achieved AUCs of 0.6186 ±0.0410 and of 0.7501 ±0.0674 on the former and latter classification tasks, respectively. The best results were obtained using semi-supervised learning, with the best model achieving 0.7751 and 0.8164 AUC, respectively. The results obtained show that semi-supervised learning could improve the performance of trained models in different melanoma classification tasks compared to supervised learning. Automatic deep learning-based diagnosis systems could support medical professionals in their decision, serving as a second opinion or as a triage tool for medical centers.

Leveraging AI-Generated Content for Synthetic Electronic Health Record Generation With Deep Learning-Based Diagnosis Model

Leveraging AI-Generated Content for Synthetic Electronic Health Record Generation With Deep Learning-Based Diagnosis Model

Diagnosis of musculoskeletal abnormalities based on improved lightweight network for multiple model fusion.

This paper introduces a solution to address the intricacy of the model employed in the deep learning-based diagnosis of musculoskeletal abnormalities and the limitations observed in the performance of a single deep learning network model. The proposed approach involves the integration of an improved EfficientNet-B2 model with MobileNetV2, resulting in the creation of FusionNet. First, EfficientNet-B2 is combined with coordinate attention (CA) to obtain CA-EfficientNet-B2. Furthermore, aiming to minimize the model parameter count, we further enhanced the mobile inverted residual bottleneck convolution module (MBConv) employed for feature extraction in EfficientNet-B2, resulting in the development of CA-MBC-EfficientNet-B2. Next, the features extracted from CA-MBC-EfficientNet-B2 and MobileNetV2 are fused. Finally, the final diagnosis of musculoskeletal abnormalities was performed by using fully connected layers. The experimental results demonstrate that, first, compared to EfficientNet-B2, CA-MBC-EfficientNet-B2 not only significantly improves the diagnostic performance of musculoskeletal abnormalities, it also reduces the parameter count and storage space by 17%. Moreover, as compared to other models, FusionNet demonstrates remarkable performance in the area of anomaly diagnosis, particularly on the elbow dataset, achieving a precision of 92.93%, an AUC of 93.89% and an accuracy of 87.10%.

Deep Learning-based diagnosis of COVID-19 using Chest X-Ray Images

Deep Learning-based diagnosis of COVID-19 using Chest X-Ray Images

Deep Learning-based Diagnosis and Localization of Pneumothorax on Portable Supine Chest X-ray in Intensive and Emergency Medicine: A Retrospective Study

PurposeTo develop two deep learning-based systems for diagnosing and localizing pneumothorax on portable supine chest X-rays (SCXRs).MethodsFor this retrospective study, images meeting the following inclusion criteria were included: (1) patient age ≥ 20 years; (2) portable SCXR; (3) imaging obtained in the emergency department or intensive care unit. Included images were temporally split into training (1571 images, between January 2015 and December 2019) and testing (1071 images, between January 2020 to December 2020) datasets. All images were annotated using pixel-level labels. Object detection and image segmentation were adopted to develop separate systems. For the detection-based system, EfficientNet-B2, DneseNet-121, and Inception-v3 were the architecture for the classification model; Deformable DETR, TOOD, and VFNet were the architecture for the localization model. Both classification and localization models of the segmentation-based system shared the UNet architecture.ResultsIn diagnosing pneumothorax, performance was excellent for both detection-based (Area under receiver operating characteristics curve [AUC]: 0.940, 95% confidence interval [CI]: 0.907–0.967) and segmentation-based (AUC: 0.979, 95% CI: 0.963–0.991) systems. For images with both predicted and ground-truth pneumothorax, lesion localization was highly accurate (detection-based Dice coefficient: 0.758, 95% CI: 0.707–0.806; segmentation-based Dice coefficient: 0.681, 95% CI: 0.642–0.721). The performance of the two deep learning-based systems declined as pneumothorax size diminished. Nonetheless, both systems were similar or better than human readers in diagnosis or localization performance across all sizes of pneumothorax.ConclusionsBoth deep learning-based systems excelled when tested in a temporally different dataset with differing patient or image characteristics, showing favourable potential for external generalizability.

M-VAAL: Multimodal Variational Adversarial Active Learning for Downstream Medical Image Analysis Tasks.

Acquiring properly annotated data is expensive in the medical field as it requires experts, time-consuming protocols, and rigorous validation. Active learning attempts to minimize the need for large annotated samples by actively sampling the most informative examples for annotation. These examples contribute significantly to improving the performance of supervised machine learning models, and thus, active learning can play an essential role in selecting the most appropriate information in deep learning-based diagnosis, clinical assessments, and treatment planning. Although some existing works have proposed methods for sampling the best examples for annotation in medical image analysis, they are not task-agnostic and do not use multimodal auxiliary information in the sampler, which has the potential to increase robustness. Therefore, in this work, we propose a Multimodal Variational Adversarial Active Learning (M-VAAL) method that uses auxiliary information from additional modalities to enhance the active sampling. We applied our method to two datasets: i) brain tumor segmentation and multi-label classification using the BraTS2018 dataset, and ii) chest X-ray image classification using the COVID-QU-Ex dataset. Our results show a promising direction toward data-efficient learning under limited annotations.

Automated Deep Learning-Based Diagnosis and Molecular Characterization of Acute Myeloid Leukemia Using Flow Cytometry

The current flow cytometric analysis of blood and bone marrow samples for diagnosis of acute myeloid leukemia (AML) relies heavily on manual intervention in the processing and analysis steps, introducing significant subjectivity into resulting diagnoses and necessitating highly trained personnel. Furthermore, concurrent molecular characterization via cytogenetics and targeted sequencing can take multiple days, delaying patient diagnosis and treatment. Attention-based multi-instance learning models (ABMILMs) are deep learning models that make accurate predictions and generate interpretable insights regarding the classification of a sample from individual events/cells; nonetheless, these models have yet to be applied to flow cytometry data. In this study, we developed a computational pipeline using ABMILMs for the automated diagnosis of AML cases based exclusively on flow cytometric data. Analysis of 1820 flow cytometry samples shows that this pipeline provides accurate diagnoses of acute leukemia (area under the receiver operating characteristic curve [AUROC] 0.961) and accurately differentiates AML vs B- and T-lymphoblastic leukemia (AUROC 0.965). Models for prediction of 9 cytogenetic aberrancies and 32 pathogenic variants in AML provide accurate predictions, particularly for t(15;17)(PML::RARA) [AUROC 0.929], t(8;21)(RUNX1::RUNX1T1) (AUROC 0.814), and NPM1 variants (AUROC 0.807). Finally, we demonstrate how these models generate interpretable insights into which individual flow cytometric events and markers deliver optimal diagnostic utility, providing hematopathologists with a data visualization tool for improved data interpretation, as well as novel biological associations between flow cytometric marker expression and cytogenetic/molecular variants in AML. Our study is the first to illustrate the feasibility of using deep learning-based analysis of flow cytometric data for automated AML diagnosis and molecular characterization.

Hounsfield Unit Variations-Based Liver Lesions Detection and Classification Using Deep Learning.

Deep learning-based diagnosis systems are useful to identify abnormalities in medical images with the greatly increased workload of doctors. Specifically, the rate of new cases and deaths from malignancies is rising for liver diseases. Early detection of liver lesions plays an extremely important role in effective treatment and gives a higher chance of survival for patients. Therefore, automatic detection and classification of common liver lesions are essential for doctors. In fact, radiologists mainly rely on Hounsfield Units to locate liver lesions but previous studies often pay little attention to this factor. In this paper, we propose an improved method for the automatic classification of common liver lesions based on deep learning techniques and the variation of Hounsfield Unit densities on CT images with and without contrast. Hounsfield Unit is used to locate liver lesions accurately and support data labeling for classification. We construct a multi-phase classification model developed on the deep neural networks of Faster R-CNN, R-FCN, SSD, and Mask R-CNN with the transfer learning approach. The experiments are conducted on six scenarios with multi-phase CT images of common liver lesions. Experimental results show that the proposed method improves the detection and classification of liver lesions compared with recent methods because its accuracy achieves up to 97.4%. The proposed models are very useful to assist doctors in the automatic segmentation and classification of liver lesions to solve the problem of depending on the clinician's experience in the diagnosis and treatment of liver lesions.

A novel hybrid meta-learning for few-shot gas-insulated switchgear insulation defect diagnosis

Though deep learning-based diagnosis methods have been extensively researched in gas-insulated switchgear (GIS) insulation defects diagnosis. Training a high-precision and robust GIS insulation defect diagnosis model under complex and few-shot conditions remains difficult. To address these issues, a novel hybrid meta-learning for a few-shot GIS insulation defect diagnosis is proposed. The meta Siamese network (MSN) is designed by incorporating the model-agnostic meta-learning (MAML) into the Siamese network (SN). Firstly, a SN is designed to acquire knowledge of GIS insulation defect diagnosis. To capture the discriminative characteristics, the attention mechanism is introduced into SN to reduce the attention to irrelevant information. Then, the model parameters are optimized through MAML. To ensure the model’s optimal performance, the meta-stochastic gradient descent is introduced to realize optimizer learning in an end-to-end form. The experimental results show that when the support sets is 5, MSN can achieve 93.15% accuracy, which outperforms other methods. Furthermore, the issue of unbalanced samples is effectively avoided, providing a feasible solution for the few-shot GIS insulation defects.

Deep learning-based diagnosis of disease activity in patients with Graves’ orbitopathy using orbital SPECT/CT

PurposeOrbital [99mTc]TcDTPA orbital single-photon emission computed tomography (SPECT)/CT is an important method for assessing inflammatory activity in patients with Graves’ orbitopathy (GO). However, interpreting the results requires substantial physician workload. We aim to propose an automated method called GO-Net to detect inflammatory activity in patients with GO.Materials and methodsGO-Net had two stages: (1) a semantic V-Net segmentation network (SV-Net) that extracts extraocular muscles (EOMs) in orbital CT images and (2) a convolutional neural network (CNN) that uses SPECT/CT images and the segmentation results to classify inflammatory activity. A total of 956 eyes from 478 patients with GO (active: 475; inactive: 481) at Xiangya Hospital of Central South University were investigated. For the segmentation task, five-fold cross-validation with 194 eyes was used for training and internal validation. For the classification task, 80% of the eye data were used for training and internal fivefold cross-validation, and the remaining 20% of the eye data were used for testing. The EOM regions of interest (ROIs) were manually drawn by two readers and reviewed by an experienced physician as ground truth for segmentation GO activity was diagnosed according to clinical activity scores (CASs) and the SPECT/CT images. Furthermore, results are interpreted and visualized using gradient-weighted class activation mapping (Grad-CAM).ResultsThe GO-Net model combining CT, SPECT, and EOM masks achieved a sensitivity of 84.63%, a specificity of 83.87%, and an area under the receiver operating curve (AUC) of 0.89 (p < 0.01) on the test set for distinguishing active and inactive GO. Compared with the CT-only model, the GO-Net model showed superior diagnostic performance. Moreover, Grad-CAM demonstrated that the GO-Net model placed focus on the GO-active regions. For EOM segmentation, our segmentation model achieved a mean intersection over union (IOU) of 0.82.ConclusionThe proposed Go-Net model accurately detected GO activity and has great potential in the diagnosis of GO.

Deep learning-based diagnosis of histopathological patterns for invasive non-mucinous lung adenocarcinoma using semantic segmentation

ObjectivesThe application of artificial intelligence (AI) to the field of pathology has facilitated the development of digital pathology, hence, making AI-assisted diagnosis possible. Due to the variety of lung cancers...

Efficient identification of Alzheimer’s brain dynamics with Spatial-Temporal Autoencoder: A deep learning approach for diagnosing brain disorders

Alzheimer’s disease (AD) is a progressive neurological disorder seriously affecting cognitive and behavior abilities of the older people. Accurate and early diagnosis of AD is critical for improving the therapeutic effect and alleviate the clinical symptom. In this work, we proposed an automatic EEG-based diagnosis scheme for AD patients with deep learning methods. A Spatial-Temporal Autoencoder (STAE) framework with Convolutional Neural Network (CNN)- Long Short-Term-Memory (LSTM) generative model was designed to estimate latent factors of the observed oscillatory activity in the brain from multi-channels electroencephalogram (EEG) signals via unsupervised learning. Based on latent factor analysis, Alzheimer’s brain dynamics on single-trials were inferred and temporal evolution of latent brain state was analyzed in low-dimensional state space. The study mainly showed that: i) the brain state trajectories of AD patients were distinct from healthy subjects, resulting in different forms of ring manifolds and allowing to accurately identify AD; ii) experimental results demonstrated the efficiency and flexibility of the proposed deep learning-based diagnosis scheme, by which the classification of AD patients and the normal based on clinical EEG dataset achieved an accuracy of 96.30%, a sensitivity of 97.73% and a specificity of 94.69% with a multiple layer perception (MLP) classifier; iii) compared with other approaches for latent brain dynamics estimation, STAE exhibited superior performance with high accuracy of AD identification and strongly robust against instabilities in EEG recordings. The present results reveal the neuropathological mechanism of Alzheimer’s disease with brain dynamics variations and provide a feasible diagnosis tool for brain disorders.

Deep learning-based diagnosis of feline hypertrophic cardiomyopathy.

Feline hypertrophic cardiomyopathy (HCM) is a common heart disease affecting 10-15% of all cats. Cats with HCM exhibit breathing difficulties, lethargy, and heart murmur; furthermore, feline HCM can also result in sudden death. Among various methods and indices, radiography and ultrasound are the gold standards in the diagnosis of feline HCM. However, only 75% accuracy has been achieved using radiography alone. Therefore, we trained five residual architectures (ResNet50V2, ResNet152, InceptionResNetV2, MobileNetV2, and Xception) using 231 ventrodorsal radiographic images of cats (143 HCM and 88 normal) and investigated the optimal architecture for diagnosing feline HCM through radiography. To ensure the generalizability of the data, the x-ray images were obtained from 5 independent institutions. In addition, 42 images were used in the test. The test data were divided into two; 22 radiographic images were used in prediction analysis and 20 radiographic images of cats were used in the evaluation of the peeking phenomenon and the voting strategy. As a result, all models showed > 90% accuracy; Resnet50V2: 95.45%; Resnet152: 95.45; InceptionResNetV2: 95.45%; MobileNetV2: 95.45% and Xception: 95.45. In addition, two voting strategies were applied to the five CNN models; softmax and majority voting. As a result, the softmax voting strategy achieved 95% accuracy in combined test data. Our findings demonstrate that an automated deep-learning system using a residual architecture can assist veterinary radiologists in screening HCM.

Deep Learning-Based Diagnosis of Fatal Hypothermia Using Post-Mortem Computed Tomography.

In forensic medicine, fatal hypothermia diagnosis is not always easy because findings are not specific, especially if traumatized. Post-mortem computed tomography (PMCT) is a useful adjunct to the cause-of-death diagnosis and some qualitative image character analysis, such as diffuse hyperaeration with decreased vascularity or pulmonary emphysema, have also been utilized for fatal hypothermia. However, it is challenging for inexperienced forensic pathologists to recognize the subtle differences of fatal hypothermia in PMCT images. In this study, we developed a deep learning-based diagnosis system for fatal hypothermia and explored the possibility of being an alternative diagnostic for forensic pathologists. An in-house dataset of forensic autopsy proven samples was used for the development and performance evaluation of the deep learning system. We used the area under the receiver operating characteristic curve (AUC) of the system for evaluation, and a human-expert comparable AUC value of 0.905, sensitivity of 0.948, and specificity of 0.741 were achieved. The experimental results clearly demonstrated the usefulness and feasibility of the deep learning system for fatal hypothermia diagnosis.