Interpretable Deep Learning Model Research Articles

Interpreting deep learning models for glioma survival classification using visualization and textual explanations

BackgroundSaliency-based algorithms are able to explain the relationship between input image pixels and deep-learning model predictions. However, it may be difficult to assess the clinical value of the most important image features and the model predictions derived from the raw saliency map. This study proposes to enhance the interpretability of saliency-based deep learning model for survival classification of patients with gliomas, by extracting domain knowledge-based information from the raw saliency maps.Materials and methodsOur study includes presurgical T1-weighted (pre- and post-contrast), T2-weighted and T2-FLAIR MRIs of 147 glioma patients from the BraTs 2020 challenge dataset aligned to the SRI 24 anatomical atlas. Each image exam includes a segmentation mask and the information of overall survival (OS) from time of diagnosis (in days). This dataset was divided into training (n=118) and validation (n=29) datasets. The extent of surgical resection for all patients was gross total resection. We categorized the data into 42 short (mean mu =157 days), 30 medium (mu =369 days), and 46 long (mu =761 days) survivors. A 3D convolutional neural network (CNN) trained on brain tumour MRI volumes classified all patients based on expected prognosis of either short-term, medium-term, or long-term survival. We extend the popular 2D Gradient-weighted Class Activation Mapping (Grad-CAM), for the generation of saliency map, to 3D and combined it with the anatomical atlas, to extract brain regions, brain volume and probability map that reveal domain knowledge-based information.ResultsFor each OS class, a larger tumor volume was associated with a shorter OS. There were 10, 7 and 27 tumor locations in brain regions that uniquely associate with the short-term, medium-term, and long-term survival, respectively. Tumors located in the transverse temporal gyrus, fusiform, and palladium are associated with short, medium and long-term survival, respectively. The visual and textual information displayed during OS prediction highlights tumor location and the contribution of different brain regions to the prediction of OS. This algorithm design feature assists the physician in analyzing and understanding different model prediction stages.ConclusionsDomain knowledge-based information extracted from the saliency map can enhance the interpretability of deep learning models. Our findings show that tumors overlapping eloquent brain regions are associated with short patient survival.

Reducing the risk of hallucinations with interpretable deep learning models for low-dose CT denoising: comparative performance analysis

Objective. Reducing CT radiation dose is an often proposed measure to enhance patient safety, which, however results in increased image noise, translating into degradation of clinical image quality. Several deep learning methods have been proposed for low-dose CT (LDCT) denoising. The high risks posed by possible hallucinations in clinical images necessitate methods which aid the interpretation of deep learning networks. In this study, we aim to use qualitative reader studies and quantitative radiomics studies to assess the perceived quality, signal preservation and statistical feature preservation of LDCT volumes denoised by deep learning. We aim to compare interpretable deep learning methods with classical deep neural networks in clinical denoising performance. Approach. We conducted an image quality analysis study to assess the image quality of the denoised volumes based on four criteria to assess the perceived image quality. We subsequently conduct a lesion detection/segmentation study to assess the impact of denoising on signal detectability. Finally, a radiomic analysis study was performed to observe the quantitative and statistical similarity of the denoised images to standard dose CT (SDCT) images. Main results. The use of specific deep learning based algorithms generate denoised volumes which are qualitatively inferior to SDCT volumes(p < 0.05). Contrary to previous literature, denoising the volumes did not reduce the accuracy of the segmentation (p > 0.05). The denoised volumes, in most cases, generated radiomics features which were statistically similar to those generated from SDCT volumes (p > 0.05). Significance. Our results show that the denoised volumes have a lower perceived quality than SDCT volumes. Noise and denoising do not significantly affect detectability of the abdominal lesions. Denoised volumes also contain statistically identical features to SDCT volumes.

A Self-Attention-Based Neural Network for Predicting Immune Checkpoint Inhibitors Response.

Cancer cells evade immune system by negatively regulating T cells via immune checkpoints (e.g., PD-1). By blocking these checkpoints, the ability of immune system to recognize and kill cancer cells restores. Individual response rate of checkpoint blockade varies among patients, with 50%-80% in specific types of cancer such as melanoma, while only 15%-30% in most other tumors. Yet it is still an open question what is the set of biomarkers that are crucial to the response to immune checkpoint inhibitors (ICI). The overall goal of this study is to develop and validate a biologically-aware interpretable deep learning model to identify the biomarkers that can predict the survival outcome to ICI treatment. The self-attention mechanism could yield interpretable results where important biomarkers may have more "attention". However, in classical self-attention mechanism, the prior biological knowledge of protein interactions (PPI) and gene pathways are not incorporated. In this study, we propose a weighted biologically-aware attention score, where it is weighted against the gene centrality and pathway length. The genes that are closely connected to mutated genes receive 'high attention', while the genes that are far away from mutated genes along the pathway receive "lower attention". We then train, validate and test our model using 1,660 patients of nine types of cancer. To validate the prediction, 1. We evaluate the accuracy via concordance index. 2. We identified the genes that receive high attention and verify their functions in existed literature. 3. We perform sanity check by removing these genes from the data, re-training and predicting again, and comparing the prediction accuracy. Our framework has achieved an average accuracy (measured via c-index) of 0.60 ± 0.06 for NSCLC and 0.58 ± 0.07 for melanoma, which is superior to both the gold standard COX-PH model (0.57 ± 0.06 for NSCLC and 0.53 ± 0.03 for melanoma) and DeepSurv (0.54 ± 0.05 for NSCLC and 0.51 ± 0.10 for melanoma). Genes that receive high attention have been validated by supporting literature, which provides an additional means of verifying the prediction in comparison to "black box" deep learning models, where there is no way to comprehend the reason behind predictions. Removing the top 8% high-attention genes (∼25 genes) from the data while using the remaining 92% for making predictions resulted in a drop in accuracy to 0.55 ± 0.073 for NSCLC and 0.56 ± 0.03 for melanoma, underscoring the significance of these genes. Patient stratification is also performed by dividing patients into responders and non-responders based on prediction score. In this study, we propose and validate a biologically-aware self-attention based deep learning model which outperforms commonly-used survival models. Additionally, this tool has the potential to identify key biomarkers while assist in clinical decision-making, which demonstrates a promising step for immunotherapy response prediction.

EEG-Based Cross-Subject Driver Drowsiness Recognition With an Interpretable Convolutional Neural Network.

In the context of electroencephalogram (EEG)-based driver drowsiness recognition, it is still challenging to design a calibration-free system, since EEG signals vary significantly among different subjects and recording sessions. Many efforts have been made to use deep learning methods for mental state recognition from EEG signals. However, existing work mostly treats deep learning models as black-box classifiers, while what have been learned by the models and to which extent they are affected by the noise in EEG data are still underexplored. In this article, we develop a novel convolutional neural network combined with an interpretation technique that allows sample-wise analysis of important features for classification. The network has a compact structure and takes advantage of separable convolutions to process the EEG signals in a spatial-temporal sequence. Results show that the model achieves an average accuracy of 78.35% on 11 subjects for leave-one-out cross-subject drowsiness recognition, which is higher than the conventional baseline methods of 53.40%-72.68% and state-of-the-art deep learning methods of 71.75%-75.19%. Interpretation results indicate the model has learned to recognize biologically meaningful features from EEG signals, e.g., alpha spindles, as strong indicators of drowsiness across different subjects. In addition, we also explore reasons behind some wrongly classified samples with the interpretation technique and discuss potential ways to improve the recognition accuracy. Our work illustrates a promising direction on using interpretable deep learning models to discover meaningful patterns related to different mental states from complex EEG signals.

An artificial intelligence method for predicting postoperative urinary incontinence based on multiple anatomic parameters of MRI

BackgroundDeep learning methods are increasingly applied in the medical field; however, their lack of interpretability remains a challenge. Captum is a tool that can be used to interpret neural network models by computing feature importance weights. Although Captum is an interpretable model, it is rarely used to study medical problems, and there is a scarcity of data regarding MRI anatomical measurements for patients with prostate cancer after undergoing Robotic-Assisted Radical Prostatectomy (RARP). Consequently, predictive models for continence that use multiple types of anatomical MRI measurements are limited. MethodsWe explored the energy efficiency of deep learning models for predicting continence by analyzing MRI measurements. We analyzed and compared various statistical models and provided reference examples for the clinical application of interpretable deep-learning models. Patients who underwent RARP at our institution between July 2019 and December 2020 were included in this study. A series of clinical MRI anatomical measurements from these patients was used to discover continence features, and their impact on continence was primarily evaluated using a series of statistical methods and computational models. ResultsAge and six other anatomical measurements were identified as the top seven features of continence by the proposed model UINet7 with an accuracy of 0.97, and the first four of these features were also found by primary statistical analysis. ConclusionsThis study fills the gaps in the in-depth investigation of continence features after RARP due to the limitations of clinical data and applicable models. We provide a pioneering example of the application of deep-learning models to clinical problems. The interpretability analysis of deep learning models has the potential for clinical applications.

An interpretable deep learning model for time-series electronic health records: Case study of delirium prediction in critical care

Deep Learning (DL) models have received increasing attention in the clinical setting, particularly in intensive care units (ICU). In this context, the interpretability of the outcomes estimated by the DL models is an essential step towards increasing adoption of DL models in clinical practice. To address this challenge, we propose an ante-hoc, interpretable neural network model. Our proposed model, named double self-attention architecture (DSA), uses two attention-based mechanisms, including self-attention and effective attention. It can capture the importance of input variables in general, as well as changes in importance along the time dimension for the outcome of interest. We evaluated our model using two real-world clinical datasets covering 22840 patients in predicting onset of delirium 12 h and 48 h in advance. Additionally, we compare the descriptive performance of our model with three post-hoc interpretable algorithms as well as with the opinion of clinicians based on the published literature and clinical experience. We find that our model covers the majority of the top-10 variables ranked by the other three post-hoc interpretable algorithms as well as the clinical opinion, with the advantage of taking into account both, the dependencies among variables as well as dependencies between varying time-steps. Finally, our results show that our model can improve descriptive performance without sacrificing predictive performance.

Grading surface urban heat island and investigating factor weight based on interpretable deep learning model across global cities

Significant urbanization resulted in increasing surface urban heat island (SUHI) that caused negative impacts on urban ecological environment, and residential comfort. Accurately monitoring the spatiotemporal variations and understanding controls of SUHI were essential to propose effective mitigation measurements. However, SUHI grades across global cities remained unknown, which cloud greatly support for global mitigations. Additionally, quantitative evaluating factor weights for different SUHI indicators and grades worldwide remained further investigations. Therefore, this paper proposed SUHI grading based on agglomerative hierarchical clustering, and further quantified factor weights for different indicators and grades based on an interoperable machine learning named TabNet. There were three major findings. (1) Global cities were grouped into five grades, including SUCI (surface urban cool island), insignificant, low-value, medium-value, and high-value SUHI grades, indicating significant differences among different grades. SUHI grades showed significant climate-based variations, wherein the arid climate was dominated by the SUCI grade at daytime but the high-value grade at nighttime. (2) Vegetation difference was an important factor for daytime SUHII accounting for 27%. Daytime frequency of SUHI was controlled by vegetation difference, temperature, evaporation and nighttime light, accounting for 78%. The major factors for nighttime frequency were albedo differences and nighttime light, accounting for 45%. (3) Related factors contributed differently to various SUHI grades. The weight of △EVI for daytime SUHII gradually increased with grades, while it for daytime frequency and maximum duration of SUHI decreased with grades. The nighttime SUHII of the low-value grade was greatly affected by the background climate, while that of the medium-value and high-value grades were strongly impacted by anthropogenic heat flux. The diurnal contrast of grades and coupling effects with heat wave were further discussed. This paper aimed to provide information on grades and controls of SUHI for further mitigation proposal.

Runoff Prediction in the Xijiang River Basin Based on Long Short-Term Memory with Variant Models and Its Interpretable Analysis

The Long Short-Term Memory (LSTM) neural network model is an effective deep learning approach for predicting streamflow, and the investigation of the interpretability of deep learning models in streamflow prediction is of great significance for model transfer and improvement. In this study, four key hydrological stations in the Xijiang River Basin (XJB) in South China are taken as examples, and the performance of the LSTM model and its variant models in runoff prediction were evaluated under the same foresight period, and the impacts of different foresight periods on the prediction results were investigated based on the SHapley Additive exPlanations (SHAP) method to explore the interpretability of the LSTM model in runoff prediction. The results showed that (1) LSTM was the optimal model among the four models in the XJB; (2) the predicted results of the LSTM model decreased with the increase in foresight period, with the Nash–Sutcliffe efficiency coefficient (NSE) decreasing by 4.7% when the foresight period increased from one month to two months, and decreasing by 3.9% when the foresight period increased from two months to three months; (3) historical runoff had the greatest impact on streamflow prediction, followed by precipitation, evaporation, and the North Pacific Index (NPI); except evaporation, all the others were positively correlated. The results can provide a reference for monthly runoff prediction in the XJB.

An explainable deep-learning model to stage sleep states in children and propose novel EEG-related patterns in sleep apnea

Automatic deep-learning models used for sleep scoring in children with obstructive sleep apnea (OSA) are perceived as black boxes, limiting their implementation in clinical settings. Accordingly, we aimed to develop an accurate and interpretable deep-learning model for sleep staging in children using single-channel electroencephalogram (EEG) recordings. We used EEG signals from the Childhood Adenotonsillectomy Trial (CHAT) dataset (n = 1637) and a clinical sleep database (n = 980). Three distinct deep-learning architectures were explored to automatically classify sleep stages from a single-channel EEG data. Gradient-weighted Class Activation Mapping (Grad-CAM), an explainable artificial intelligence (XAI) algorithm, was then applied to provide an interpretation of the singular EEG patterns contributing to each predicted sleep stage. Among the tested architectures, a standard convolutional neural network (CNN) demonstrated the highest performance for automated sleep stage detection in the CHAT test set (accuracy = 86.9% and five-class kappa = 0.827). Furthermore, the CNN-based estimation of total sleep time exhibited strong agreement in the clinical dataset (intra-class correlation coefficient = 0.772). Our XAI approach using Grad-CAM effectively highlighted the EEG features associated with each sleep stage, emphasizing their influence on the CNN's decision-making process in both datasets. Grad-CAM heatmaps also allowed to identify and analyze epochs within a recording with a highly likelihood to be misclassified, revealing mixed features from different sleep stages within these epochs. Finally, Grad-CAM heatmaps unveiled novel features contributing to sleep scoring using a single EEG channel. Consequently, integrating an explainable CNN-based deep-learning model in the clinical environment could enable automatic sleep staging in pediatric sleep apnea tests.

One and one make eleven: An interpretable neural network for image recognition

Although non-interpretable (black-box) deep learning models are well known for their accuracy, interpretable deep learning models should be used for high stake decisions, such as: healthcare. In this paper, we present a novel technique of combining the existing state-of-the-art models, and using them as a base model to build an interpretable deep learning model Comb-ProtoPNet. In contrast to usual technique of combining the logits of two (or more) algorithms to form an ensemble algorithm, we combine the algorithms themselves. Our proposed interpretable model applies a prototype layer on top of the convolutional layers of an ensemble base model. We trained and tested our algorithm over the dataset of chest CT-scan images of COVID-19 patients, pneumonia patients and normal people. The use of a certain combination of blocks from two different state-of-the-art models (statistically) significantly improved the accuracy compared to the individual use of the state-of-the-art models as the base models, and this is where the part of the title “One and one make eleven” comes from.

Deciphering the connection between upstream obstacles, wake structures, and root signals in seal whisker array sensing using interpretable neural networks.

This study presents a novel method that combines a computational fluid-structure interaction model with an interpretable deep-learning model to explore the fundamental mechanisms of seal whisker sensing. By establishing connections between crucial signal patterns, flow characteristics, and attributes of upstream obstacles, the method has the potential to enhance our understanding of the intricate sensing mechanisms. The effectiveness of the method is demonstrated through its accurate prediction of the location and orientation of a circular plate placed in front of seal whisker arrays. The model also generates temporal and spatial importance values of the signals, enabling the identification of significant temporal-spatial signal patterns crucial for the network's predictions. These signal patterns are further correlated with flow structures, allowing for the identification of important flow features relevant for accurate prediction. The study provides insights into seal whiskers' perception of complex underwater environments, inspiring advancements in underwater sensing technologies.

XMR: an explainable multimodal neural network for drug response prediction.

Introduction: Existing large-scale preclinical cancer drug response databases provide us with a great opportunity to identify and predict potentially effective drugs to combat cancers. Deep learning models built on these databases have been developed and applied to tackle the cancer drug-response prediction task. Their prediction has been demonstrated to significantly outperform traditional machine learning methods. However, due to the "black box" characteristic, biologically faithful explanations are hardly derived from these deep learning models. Interpretable deep learning models that rely on visible neural networks (VNNs) have been proposed to provide biological justification for the predicted outcomes. However, their performance does not meet the expectation to be applied in clinical practice. Methods: In this paper, we develop an XMR model, an eXplainable Multimodal neural network for drug Response prediction. XMR is a new compact multimodal neural network consisting of two sub-networks: a visible neural network for learning genomic features and a graph neural network (GNN) for learning drugs' structural features. Both sub-networks are integrated into a multimodal fusion layer to model the drug response for the given gene mutations and the drug's molecular structures. Furthermore, a pruning approach is applied to provide better interpretations of the XMR model. We use five pathway hierarchies (cell cycle, DNA repair, diseases, signal transduction, and metabolism), which are obtained from the Reactome Pathway Database, as the architecture of VNN for our XMR model to predict drug responses of triple negative breast cancer. Results: We find that our model outperforms other state-of-the-art interpretable deep learning models in terms of predictive performance. In addition, our model can provide biological insights into explaining drug responses for triple-negative breast cancer. Discussion: Overall, combining both VNN and GNN in a multimodal fusion layer, XMR captures key genomic and molecular features and offers reasonable interpretability in biology, thereby better predicting drug responses in cancer patients. Our model would also benefit personalized cancer therapy in the future.

Revealing the Impact of Genomic Alterations on Cancer Cell Signaling with an Interpretable Deep Learning Model.

Cancer is a disease of aberrant cellular signaling resulting from somatic genomic alterations (SGAs). Heterogeneous SGA events in tumors lead to tumor-specific signaling system aberrations. We interpret the cancer signaling system as a causal graphical model, where SGAs affect signaling proteins, propagate their effects through signal transduction, and ultimately change gene expression. To represent such a system, we developed a deep learning model called redundant-input neural network (RINN) with a transparent redundant-input architecture. Our findings demonstrate that by utilizing SGAs as inputs, the RINN can encode their impact on the signaling system and predict gene expression accurately when measured as the area under ROC curves. Moreover, the RINN can discover the shared functional impact (similar embeddings) of SGAs that perturb a common signaling pathway (e.g., PI3K, Nrf2, and TGF). Furthermore, the RINN exhibits the ability to discover known relationships in cellular signaling systems.

Assisting schizophrenia diagnosis using clinical electroencephalography and interpretable graph neural networks: a real-world and cross-site study.

Schizophrenia (SCZ) is a chronic and serious mental disorder with a high mortality rate. At present, there is a lack of objective, cost-effective and widely disseminated diagnosis tools to address this mental health crisis globally. Clinical electroencephalogram (EEG) is a noninvasive technique to measure brain activity with high temporal resolution, and accumulating evidence demonstrates that clinical EEG is capable of capturing abnormal SCZ neuropathology. Although EEG-based automated diagnostic tools have obtained impressive performance on individual datasets, the transportability of potential EEG biomarkers in cross-site real-world application is still an open question. To address the challenges of small sample sizes and population heterogeneity, we develop an advanced interpretable deep learning model using multimodal clinical EEG features and demographic information as inputs to graph neural networks, and further propose different transfer learning strategies to adapt to different clinical scenarios. Taking the disease discrimination of health control (HC) and SCZ with 1030 participants as a use case, our model is trained on a small clinical dataset (N = 188, Chinese) and enhanced using a large-scale public dataset (N = 508, American) of adult participants. Cross-site validation from an independent dataset of adult participants (N = 157, Chinese) produced stable performance, with AUCs of 0.793-0.852 and accuracies of 0.786-0.858 for different SCZ prevalence, respectively. In addition, cross-site validation from another dataset of adolescent boys (N = 84, Russian) yielded an AUC of 0.702 and an accuracy of 0.690. Moreover, feature visualization further revealed that the ranking of feature importance varied significantly among different datasets, and that EEG theta and alpha band power appeared to be the most significant and translational biomarkers of SCZ pathology. Overall, our promising results demonstrate the feasibility of SCZ discrimination using EEG biomarkers in multiple clinical settings.

Knowledge graph aids comprehensive explanation of drug and chemical toxicity.

In computational toxicology, prediction of complex endpoints has always been challenging, as they often involve multiple distinct mechanisms. State-of-the-art models are either limited by low accuracy, or lack of interpretability due to their black-box nature. Here, we introduce AIDTox, an interpretable deep learning model which incorporates curated knowledge of chemical-gene connections, gene-pathway annotations, and pathway hierarchy. AIDTox accurately predicts cytotoxicity outcomes in HepG2 and HEK293 cells. It also provides comprehensive explanations of cytotoxicity covering multiple aspects of drug activity, including target interaction, metabolism, and elimination. In summary, AIDTox provides a computational framework for unveiling cellular mechanisms for complex toxicity endpoints.

PINNet: a deep neural network with pathway prior knowledge for Alzheimer's disease.

Identification of Alzheimer's Disease (AD)-related transcriptomic signatures from blood is important for early diagnosis of the disease. Deep learning techniques are potent classifiers for AD diagnosis, but most have been unable to identify biomarkers because of their lack of interpretability. To address these challenges, we propose a pathway information-based neural network (PINNet) to predict AD patients and analyze blood and brain transcriptomic signatures using an interpretable deep learning model. PINNet is a deep neural network (DNN) model with pathway prior knowledge from either the Gene Ontology or Kyoto Encyclopedia of Genes and Genomes databases. Then, a backpropagation-based model interpretation method was applied to reveal essential pathways and genes for predicting AD. The performance of PINNet was compared with a DNN model without a pathway. Performances of PINNet outperformed or were similar to those of DNN without a pathway using blood and brain gene expressions, respectively. Moreover, PINNet considers more AD-related genes as essential features than DNN without a pathway in the learning process. Pathway analysis of protein-protein interaction modules of highly contributed genes showed that AD-related genes in blood were enriched with cell migration, PI3K-Akt, MAPK signaling, and apoptosis in blood. The pathways enriched in the brain module included cell migration, PI3K-Akt, MAPK signaling, apoptosis, protein ubiquitination, and t-cell activation. By integrating prior knowledge about pathways, PINNet can reveal essential pathways related to AD. The source codes are available at https://github.com/DMCB-GIST/PINNet.

The State of the Art in Deep Learning Applications, Challenges, and Future Prospects: A Comprehensive Review of Flood Forecasting and Management

Floods are a devastating natural calamity that may seriously harm both infrastructure and people. Accurate flood forecasts and control are essential to lessen these effects and safeguard populations. By utilizing its capacity to handle massive amounts of data and provide accurate forecasts, deep learning has emerged as a potent tool for improving flood prediction and control. The current state of deep learning applications in flood forecasting and management is thoroughly reviewed in this work. The review discusses a variety of subjects, such as the data sources utilized, the deep learning models used, and the assessment measures adopted to judge their efficacy. It assesses current approaches critically and points out their advantages and disadvantages. The article also examines challenges with data accessibility, the interpretability of deep learning models, and ethical considerations in flood prediction. The report also describes potential directions for deep-learning research to enhance flood predictions and control. Incorporating uncertainty estimates into forecasts, integrating many data sources, developing hybrid models that mix deep learning with other methodologies, and enhancing the interpretability of deep learning models are a few of these. These research goals can help deep learning models become more precise and effective, which will result in better flood control plans and forecasts. Overall, this review is a useful resource for academics and professionals working on the topic of flood forecasting and management. By reviewing the current state of the art, emphasizing difficulties, and outlining potential areas for future study, it lays a solid basis. Communities may better prepare for and lessen the destructive effects of floods by implementing cutting-edge deep learning algorithms, thereby protecting people and infrastructure.

A self-supervised vision transformer to predict survival from histopathology in renal cell carcinoma

PurposeTo develop and validate an interpretable deep learning model to predict overall and disease-specific survival (OS/DSS) in clear cell renal cell carcinoma (ccRCC).MethodsDigitised haematoxylin and eosin-stained slides from The Cancer Genome Atlas were used as a training set for a vision transformer (ViT) to extract image features with a self-supervised model called DINO (self-distillation with no labels). Extracted features were used in Cox regression models to prognosticate OS and DSS. Kaplan–Meier for univariable evaluation and Cox regression analyses for multivariable evaluation of the DINO-ViT risk groups were performed for prediction of OS and DSS. For validation, a cohort from a tertiary care centre was used.ResultsA significant risk stratification was achieved in univariable analysis for OS and DSS in the training (n = 443, log rank test, p < 0.01) and validation set (n = 266, p < 0.01). In multivariable analysis, including age, metastatic status, tumour size and grading, the DINO-ViT risk stratification was a significant predictor for OS (hazard ratio [HR] 3.03; 95%-confidence interval [95%-CI] 2.11–4.35; p < 0.01) and DSS (HR 4.90; 95%-CI 2.78–8.64; p < 0.01) in the training set but only for DSS in the validation set (HR 2.31; 95%-CI 1.15–4.65; p = 0.02). DINO-ViT visualisation showed that features were mainly extracted from nuclei, cytoplasm, and peritumoural stroma, demonstrating good interpretability.ConclusionThe DINO-ViT can identify high-risk patients using histological images of ccRCC. This model might improve individual risk-adapted renal cancer therapy in the future.

Interpretable deep learning architectures for improving drug response prediction performance: myth or reality?

Interpretable deep learning (DL) models that can provide biological insights, in addition to accurate predictions, are of great interest to the biomedical community. Recently, interpretable DL models that incorporate signaling pathways have been proposed for drug response prediction (DRP). While these models improve interpretability, it is unclear whether this comes at the cost of less accurate DRPs, or a prediction improvement can also be obtained. We comprehensively and systematically assessed four state-of-the-art interpretable DL models using three pathway collections to assess their ability in making accurate predictions on unseen samples from the same dataset, as well as their generalizability to an independent dataset. Our results showed that models that explicitly incorporate pathway information in the form of a latent layer perform worse compared to models that incorporate this information implicitly. However, in most evaluation setups, the best performance was achieved using a black-box multilayer perceptron, and the performance of a random forests baseline was comparable to those of the interpretable models. Replacing the signaling pathways with randomly generated pathways showed a comparable performance for the majority of the models. Finally, the performance of all models deteriorated when applied to an independent dataset. These results highlight the importance of systematic evaluation of newly proposed models using carefully selected baselines. We provide different evaluation setups and baseline models that can be used to achieve this goal. Implemented models and datasets are provided at https://doi.org/10.5281/zenodo.7787178 and https://doi.org/10.5281/zenodo.7101665, respectively.

Classification and structural characteristics of amorphous materials based on interpretable deep learning

Defining the structure characteristics of amorphous materials is one of the fundamental problems that need to be solved urgently in complex materials because of their complex structure and long-range disorder. In this study, we develop an interpretable deep learning model capable of accurately classifying amorphous configurations and characterizing their structural properties. The results demonstrate that the multi-dimensional hybrid convolutional neural network can classify the two-dimensional (2D) liquids and amorphous solids of molecular dynamics simulation. The classification process does not make a priori assumptions on the amorphous particle environment, and the accuracy is 92.75%, which is better than other convolutional neural networks. Moreover, our model utilizes the gradient-weighted activation-like mapping method, which generates activation-like heat maps that can precisely identify important structures in the amorphous configuration maps. We obtain an order parameter from the heatmap and conduct finite scale analysis of this parameter. Our findings demonstrate that the order parameter effectively captures the amorphous phase transition process across various systems. These results hold significant scientific implications for the study of amorphous structural characteristics via deep learning.