Interpretable Deep Learning Model Research Articles

O-221 An interpretable deep learning model for predicting the number of mature oocytes retrieved after ovarian hyperstimulation

Abstract Study question Can an interpretable deep learning model successfully predict the number of mature oocytes retrieved after ovarian hyperstimulation and provide the rationale for these predictions? Summary answer An interpretable deep learning model can successfully predict the number of mature oocytes retrieved and provide a rationale for these predictions. What is known already A number of recent studies have used classical machine learning models in order to predict the number of mature oocytes retrieved in IVF cycles, including XGBoost, RandomForests and even basic deep learning models such as feed forward networks. Although these studies predicted the number of mature oocytes with a relatively high performance, the employed methods lack insights into the rationale governing their predictions. This lack of interpretability poses an important limitation when applying these models in a clinical setting. Study design, size, duration This study was a retrospective analysis of a dataset of 6,430 ovarian stimulation cycles performed at a single center in the USA, aimed at building a deep learning based, interpretable model to predict the number of mature oocytes retrieved after ovarian hyperstimulation. Participants/materials, setting, methods An interpretable deep learning model was developed to predict the number of mature oocytes retrieved. The model was evaluated on an out-of-sample test set and compared to baseline models used in prior studies such as XGBoost and Linear Regression. Main results and the role of chance The model successfully predicted the retrieved oocyte count, with a mean absolute error (MAE) deviation of ± 3.14 oocytes from the ground truth. The model outperformed the performance of models used in previous studies such as XGBoost and Linear Regression which achieved a MAE of ± 3.16 and ±3.2 oocytes respectively on the test dataset. Parameter significance was measured by calculating how much parameters aided in the final prediction. The parameters found to be most significant in predicting the number of retrieved mature oocytes were (in order): total number of follicles, number of follicles size 12mm-15mm and number of follicles size 16-17mm. Parameters that were found to be of lower significance but still indicative were follicles larger than 18 mm and estradiol on trigger day. Surprisingly, age and basal FSH were of little significance. Most notably, the number of follicles in the 12mm -15mm bin were much more indicative than larger follicles of size 16-17mm and much more than follicles larger than 18mm. Furthermore, the model generates a transparent breakdown of the prediction for each patient, including a heat map and bar graph highlighting the factors influencing the decision making process at every step, creating a completely transparent prediction process. Limitations, reasons for caution The model was developed on training data from a single clinic, therefore requiring further data collection from diverse global locations to ensure the model’s ability to generalize. Wider implications of the findings This study demonstrates better oocyte prediction accuracy and transparency in the prediction process, surpassing previous models while also building trust. These advancements not only improve the safety of Artificial Intelligence tools for optimizing stimulation protocols but also contribute to reducing regulatory concerns surrounding their deployment in clinical settings. Trial registration number not applicable

Explaining the Mechanism of Multiscale Groundwater Drought Events: A New Perspective From Interpretable Deep Learning Model

AbstractThis study presents a new approach to understand the causes of groundwater drought events with interpretable deep learning (DL) models. As prerequisites, accurate long short‐term memory (LSTM) models for simulating groundwater are built for 16 regions representing three types of spatial scales in the southeastern United States, and standardized groundwater index is applied to identify 233 groundwater drought events. Two interpretation methods, expected gradients (EG) and additive decomposition (AD), are adopted to decipher the DL‐captured patterns and inner workings of LSTM networks. The EG results show that: (a) temperature‐related features were the primary drivers of large‐scale groundwater droughts, with their importance increasing from 56.1% to 63.1% as the drought events approached from 6 months to 15 days. Conversely, precipitation‐related features were found to be the dominant factors in the formation of groundwater drought in small‐scale catchments, with the overall importance ranging from 59.8% to 53.3%; (b) Seasonal variations in the importance of temperature‐related factors are inversely related between large and small spatial scales, being more significant in summer for larger regions and in winter for catchments; and (c) temperature‐related factors exhibited an overall “trigger effect” on causing groundwater drought events in the studying areas. The AD method unveiled how the LSTM network behaved differently in retaining and discarding information when emulating different groundwater droughts. In summary, this study provides a new perspective for the causes of groundwater drought events and highlights the potential and prospect of interpretable DL in enhancing our understanding of hydrological processes.

Interpretable deep learning in single-cell omics.

Single-cell omics technologies have enabled the quantification of molecular profiles in individual cells at an unparalleled resolution. Deep learning, a rapidly evolving sub-field of machine learning, has instilled a significant interest in single-cell omics research due to its remarkable success in analysing heterogeneous high-dimensional single-cell omics data. Nevertheless, the inherent multi-layer nonlinear architecture of deep learning models often makes them 'black boxes' as the reasoning behind predictions is often unknown and not transparent to the user. This has stimulated an increasing body of research for addressing the lack of interpretability in deep learning models, especially in single-cell omics data analyses, where the identification and understanding of molecular regulators are crucial for interpreting model predictions and directing downstream experimental validations. In this work, we introduce the basics of single-cell omics technologies and the concept of interpretable deep learning. This is followed by a review of the recent interpretable deep learning models applied to various single-cell omics research. Lastly, we highlight the current limitations and discuss potential future directions. Supplementary data are available at Bioinformatics online.

S2MRI-ADNet: an interpretable deep learning framework integrating Euclidean-graph representations of Alzheimer's disease solely from structural MRI.

To establish a multi-dimensional representation solely on structural MRI (sMRI) for early diagnosis of AD. A total of 3377 participants' sMRI from four independent databases were retrospectively identified to construct an interpretable deep learning model that integrated multi-dimensional representations of AD solely on sMRI (called s2MRI-ADNet) by a dual-channel learning strategy of gray matter volume (GMV) from Euclidean space and the regional radiomics similarity network (R2SN) from graph space. Specifically, the GMV feature map learning channel (called GMV-Channel) was to take into consideration spatial information of both long-range spatial relations and detailed localization information, while the node feature and connectivity strength learning channel (called NFCS-Channel) was to characterize the graph-structured R2SN network by a separable learning strategy. The s2MRI-ADNet achieved a superior classification accuracy of 92.1% and 91.4% under intra-database and inter-database cross-validation. The GMV-Channel and NFCS-Channel captured complementary group-discriminative brain regions, revealing a complementary interpretation of the multi-dimensional representation of brain structure in Euclidean and graph spaces respectively. Besides, the generalizable and reproducible interpretation of the multi-dimensional representation in capturing complementary group-discriminative brain regions revealed a significant correlation between the four independent databases (p < 0.05). Significant associations (p < 0.05) between attention scores and brain abnormality, between classification scores and clinical measure of cognitive ability, CSF biomarker, metabolism, and genetic risk score also provided solid neurobiological interpretation. The s2MRI-ADNet solely on sMRI could leverage the complementary multi-dimensional representations of AD in Euclidean and graph spaces, and achieved superior performance in the early diagnosis of AD, facilitating its potential in both clinical translation and popularization.

Explainable Deep Learning Model for Predicting Serious Adverse Events in Hospitalized Geriatric Patients Within 72 Hours.

The global aging population presents a significant challenge, with older adults experiencing declining physical and cognitive abilities and increased vulnerability to chronic diseases and adverse health outcomes. This study aims to develop an interpretable deep learning (DL) model to predict adverse events in geriatric patients within 72 hours of hospitalization. The study used retrospective data (2017-2020) from a major medical center in Taiwan. It included non-trauma geriatric patients who visited the emergency department and were admitted to the general ward. Data preprocessing involved collecting prognostic factors like vital signs, lab results, medical history, and clinical management. A deep feedforward neural network was developed, and performance was evaluated using accuracy, sensitivity, specificity, positive predictive value (PPV), and area under the receiver operating characteristic curve (AUC). Model interpretation utilized the Shapley Additive Explanation (SHAP) technique. The analysis included 127,268 patients, with 2.6% experiencing imminent intensive care unit transfer, respiratory failure, or death during hospitalization. The DL model achieved AUCs of 0.86 and 0.84 in the validation and test sets, respectively, outperforming the Sequential Organ Failure Assessment (SOFA) score. Sensitivity and specificity values ranged from 0.79 to 0.81. The SHAP technique provided insights into feature importance and interactions. The developed DL model demonstrated high accuracy in predicting serious adverse events in geriatric patients within 72 hours of hospitalization. It outperformed the SOFA score and provided valuable insights into the model's decision-making process.

Estimation of the shear strength of UHPC beams via interpretable deep learning models: Comparison of different optimization techniques

In this article, optimized deep learning (DL) models with different algorithms are adopted to estimate the shear strength of rectangular ultra-high performance concrete beams (UHPC-Bs) in order to overcome the challenges in traditional mechanics-based approaches. Long short-term memory (LSTM) and gated recurrent unit (GRU) are chosen as the DL models, whereas the recent popular optimization algorithms are phasor particle swarm optimization (PPSO), dwarf mongoose optimization (DMO), mountain gazelle optimizer (MGO), and atom search optimization (ASO). A thorough and reliable dataset of 244 UHPC-Bs test results with ten input features has been used to construct the hybrid DL models. The performance of the optimized hybrid LSTM and GRU models with different algorithms is extensively assessed and compared based on various statistical metrics, error, and score analyses. Then, the model with the best estimation performance is determined and compared with the mechanics-based formulas in the current international design codes. Additionally, Shapley additive explanations (SHAP) analysis is used to assist in the interpretability of DL models and to reveal the effects of input features that contribute to the model's estimation. According to the results of the present work, all DL models successfully estimate the shear strength of UHPC-Bs. Among these models, the MGO-LSTM model stands out compared to the other models in terms of several performance measures for both the training and testing phases, like a higher R2 value, lower RMSE, MAPE, and MAE values, as well as a smaller error ratio and a higher final score. The performance of the algorithms applied to optimize the hyper-parameters of the LSTM and GRU models can be ranked as follows: MGO > DMO > PPSO > ASO. Moreover, a graphical user interface (GUI) was constructed based on the best estimation model that was built so that the shear strength of UHPC-Bs could be estimated in real-world situations without the need for any extra software or tools. This enables more users to quickly and easily estimate the shear strength of UHPC-Bs, optimize design processes, and decrease experimental testing costs.

Deep learning approaches for breast cancer detection in histopathology images: A review.

Breast cancer is one of the leading causes of death in women worldwide. Histopathology analysis of breast tissue is an essential tool for diagnosing and staging breast cancer. In recent years, there has been a significant increase in research exploring the use of deep-learning approaches for breast cancer detection from histopathology images. To provide an overview of the current state-of-the-art technologies in automated breast cancer detection in histopathology images using deep learning techniques. This review focuses on the use of deep learning algorithms for the detection and classification of breast cancer from histopathology images. We provide an overview of publicly available histopathology image datasets for breast cancer detection. We also highlight the strengths and weaknesses of these architectures and their performance on different histopathology image datasets. Finally, we discuss the challenges associated with using deep learning techniques for breast cancer detection, including the need for large and diverse datasets and the interpretability of deep learning models. Deep learning techniques have shown great promise in accurately detecting and classifying breast cancer from histopathology images. Although the accuracy levels vary depending on the specific data set, image pre-processing techniques, and deep learning architecture used, these results highlight the potential of deep learning algorithms in improving the accuracy and efficiency of breast cancer detection from histopathology images. This review has presented a thorough account of the current state-of-the-art techniques for detecting breast cancer using histopathology images. The integration of machine learning and deep learning algorithms has demonstrated promising results in accurately identifying breast cancer from histopathology images. The insights gathered from this review can act as a valuable reference for researchers in this field who are developing diagnostic strategies using histopathology images. Overall, the objective of this review is to spark interest among scholars in this complex field and acquaint them with cutting-edge technologies in breast cancer detection using histopathology images.

CardioDPi: An explainable deep-learning model for identifying cardiotoxic chemicals targeting hERG, Cav1.2, and Nav1.5 channels

The cardiotoxic effects of various pollutants have been a growing concern in environmental and material science. These effects encompass arrhythmias, myocardial injury, cardiac insufficiency, and pericardial inflammation. Compounds such as organic solvents and air pollutants disrupt the potassium, sodium, and calcium ion channels cardiac cell membranes, leading to the dysregulation of cardiac function. However, current cardiotoxicity models have disadvantages of incomplete data, ion channels, interpretability issues, and inability of toxic structure visualization. Herein, an interpretable deep-learning model known as CardioDPi was developed, which is capable of discriminating cardiotoxicity induced by the human Ether-à-go-go-related gene (hERG) channel, sodium channel (Na_v1.5), and calcium channel (Ca_v1.5) blockade. External validation yielded promising area under the ROC curve (AUC) values of 0.89, 0.89, and 0.94 for the hERG, Na_v1.5, and Ca_v1.5 channels, respectively. The CardioDPi can be freely accessed on the web server CardioDPipredictor (http://cardiodpi.sapredictor.cn/). Furthermore, the structural characteristics of cardiotoxic compounds were analyzed and structural alerts (SAs) can be extracted using the user-friendly CardioDPi-SAdetector web service (http://cardiosa.sapredictor.cn/). CardioDPi is a valuable tool for identifying cardiotoxic chemicals that are environmental and health risks. Moreover, the SA system provides essential insights for mode-of-action studies concerning cardiotoxic compounds.

Improving Clinician Performance in Classifying EEG Patterns on the Ictal-Interictal Injury Continuum Using Interpretable Machine Learning.

In intensive care units (ICUs), critically ill patients are monitored with electroencephalography (EEG) to prevent serious brain injury. EEG monitoring is constrained by clinician availability, and EEG interpretation can be subjective and prone to interobserver variability. Automated deep-learning systems for EEG could reduce human bias and accelerate the diagnostic process. However, existing uninterpretable (black-box) deep-learning models are untrustworthy, difficult to troubleshoot, and lack accountability in real-world applications, leading to a lack of both trust and adoption by clinicians. We developed an interpretable deep-learning system that accurately classifies six patterns of potentially harmful EEG activity - seizure, lateralized periodic discharges (LPDs), generalized periodic discharges (GPDs), lateralized rhythmic delta activity (LRDA), generalized rhythmic delta activity (GRDA), and other patterns - while providing faithful case-based explanations of its predictions. The model was trained on 50,697 total 50-second continuous EEG samples collected from 2711 patients in the ICU between July 2006 and March 2020 at Massachusetts General Hospital. EEG samples were labeled as one of the six EEG patterns by 124 domain experts and trained annotators. To evaluate the model, we asked eight medical professionals with relevant backgrounds to classify 100 EEG samples into the six pattern categories - once with and once without artificial intelligence (AI) assistance - and we assessed the assistive power of this interpretable system by comparing the diagnostic accuracy of the two methods. The model's discriminatory performance was evaluated with area under the receiver-operating characteristic curve (AUROC) and area under the precision-recall curve. The model's interpretability was measured with task-specific neighborhood agreement statistics that interrogated the similarities of samples and features. In a separate analysis, the latent space of the neural network was visualized by using dimension reduction techniques to examine whether the ictal-interictal injury continuum hypothesis, which asserts that seizures and seizure-like patterns of brain activity lie along a spectrum, is supported by data. The performance of all users significantly improved when provided with AI assistance. Mean user diagnostic accuracy improved from 47 to 71% (P<0.04). The model achieved AUROCs of 0.87, 0.93, 0.96, 0.92, 0.93, and 0.80 for the classes seizure, LPD, GPD, LRDA, GRDA, and other patterns, respectively. This performance was significantly higher than that of a corresponding uninterpretable black-box model (with P<0.0001). Videos traversing the ictal-interictal injury manifold from dimension reduction (a two-dimensional representation of the original high-dimensional feature space) give insight into the layout of EEG patterns within the network's latent space and illuminate relationships between EEG patterns that were previously hypothesized but had not yet been shown explicitly. These results indicate that the ictal-interictal injury continuum hypothesis is supported by data. Users showed significant pattern classification accuracy improvement with the assistance of this interpretable deep-learning model. The interpretable design facilitates effective human-AI collaboration; this system may improve diagnosis and patient care in clinical settings. The model may also provide a better understanding of how EEG patterns relate to each other along the ictal-interictal injury continuum. (Funded by the National Science Foundation, National Institutes of Health, and others.).

An interpretable deep learning model predicts RNA–small molecule binding sites

The intricate interplay between RNA molecules and small ligands plays a pivotal role in regulating biological processes, underscoring the necessity for accurate prediction models of RNA–small molecule binding sites. In this study, we develop CapBind, a novel interpretive deep learning framework utilizing a Capsule Network architecture, to precisely identify RNA–small molecule binding sites. This framework stands out for its multifaceted approach, which leverages both the sequence and structural information of RNA as input features, allowing for a thorough understanding of the complex interactions between RNA and small molecules. Leveraging multi-source bioinformatics data, including the distribution of RNA secondary structures, our model characterizes sequence and structural features with unparalleled depth. Through a unified alignment of these multi-source feature representations, we achieve dimensional homogeneity in the potential feature maps for both types of biological data. The crux of our methodology lies in the deployment of a multi-level Capsule Network, engineered to identify the nuanced interactions of RNA sequences with small molecules. The essence of our approach resides in the implementation of a multi-layered Capsule Network. Utilizing a sophisticated routing mechanism, this network demonstrates unparalleled efficacy in learning and identifying the intricate interactions between RNA sequences and small molecules. This mechanism enables dynamic adjustment of information flow across multiple levels, thereby capturing subtle features and patterns that might elude conventional neural network architectures. The experimental results demonstrate the superior performance of our model, particularly highlighting its progress in evaluation metrics compared to current state-of-the-art models. Case studies further validate the model’s precision in identifying binding sites, offering potential insights into gene expression pathways and novel therapeutic avenues for neurodegenerative diseases. Moreover, an extensive interpretability analysis of our silicon model reveals the internal interaction mechanisms between specific RNA regions and small molecules, contributing valuable perspectives for drug design and molecular biology research. These findings underscore the model’s predictive accuracy and biological applicability, establishing our approach as a significant leap forward in the computational prediction of RNA–small molecule binding sites.

Prediction of anticancer drug sensitivity using an interpretable model guided by deep learning

BackgroundThe prediction of drug sensitivity plays a crucial role in improving the therapeutic effect of drugs. However, testing the effectiveness of drugs is challenging due to the complex mechanism of drug reactions and the lack of interpretability in most machine learning and deep learning methods. Therefore, it is imperative to establish an interpretable model that receives various cell line and drug feature data to learn drug response mechanisms and achieve stable predictions between available datasets.ResultsThis study proposes a new and interpretable deep learning model, DrugGene, which integrates gene expression, gene mutation, gene copy number variation of cancer cells, and chemical characteristics of anticancer drugs to predict their sensitivity. This model comprises two different branches of neural networks, where the first involves a hierarchical structure of biological subsystems that uses the biological processes of human cells to form a visual neural network (VNN) and an interpretable deep neural network for human cancer cells. DrugGene receives genotype input from the cell line and detects changes in the subsystem states. We also employ a traditional artificial neural network (ANN) to capture the chemical structural features of drugs. DrugGene generates final drug response predictions by combining VNN and ANN and integrating their outputs into a fully connected layer. The experimental results using drug sensitivity data extracted from the Cancer Drug Sensitivity Genome Database and the Cancer Treatment Response Portal v2 reveal that the proposed model is better than existing prediction methods. Therefore, our model achieves higher accuracy, learns the reaction mechanisms between anticancer drugs and cell lines from various features, and interprets the model’s predicted results.ConclusionsOur method utilizes biological pathways to construct neural networks, which can use genotypes to monitor changes in the state of network subsystems, thereby interpreting the prediction results in the model and achieving satisfactory prediction accuracy. This will help explore new directions in cancer treatment. More available code resources can be downloaded for free from GitHub (https://github.com/pangweixiong/DrugGene).

Pseudo-class part prototype networks for interpretable breast cancer classification

Interpretability in machine learning has become increasingly important as machine learning is being used in more and more applications, including those with high-stakes consequences such as healthcare where Interpretability has been regarded as a key to the successful adoption of machine learning models. However, using confounding/irrelevant information in making predictions by deep learning models, even the interpretable ones, poses critical challenges to their clinical acceptance. That has recently drawn researchers’ attention to issues beyond the mere interpretation of deep learning models. In this paper, we first investigate application of an inherently interpretable prototype-based architecture, known as ProtoPNet, for breast cancer classification in digital pathology and highlight its shortcomings in this application. Then, we propose a new method that uses more medically relevant information and makes more accurate and interpretable predictions. Our method leverages the clustering concept and implicitly increases the number of classes in the training dataset. The proposed method learns more relevant prototypes without any pixel-level annotated data. To have a more holistic assessment, in addition to classification accuracy, we define a new metric for assessing the degree of interpretability based on the comments of a group of skilled pathologists. Experimental results on the BreakHis dataset show that the proposed method effectively improves the classification accuracy and interpretability by respectively 8%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$8 \\%$$\\end{document} and 18%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$18 \\%$$\\end{document}. Therefore, the proposed method can be seen as a step toward implementing interpretable deep learning models for the detection of breast cancer using histopathology images.

Toward better atmospheric polycyclic aromatic hydrocarbons pollution control in the Northern Hemisphere: Process analysis based on interpretable deep learning models

The accurate prediction of atmospheric polycyclic aromatic hydrocarbons (PAHs) concentrations in the Arctic is crucial in guiding the development of pollution control measures for atmospheric PAHs. However, the complex formation process and strong time-dependence of PAHs in the Arctic increase the difficulty of prediction. Herein, deep learning (DL) models were developed to predict monthly air concentrations of 16 priority PAHs at Alert monitoring station by incorporating the inherent periodicity of PAH concentrations and ancillary data including PAH emissions, meteorological information, fire emissions, and sea ice area. The prediction performance of long short-term memory (LSTM), gated recurrent unit (GRU) and bidirectional LSTM (Bi LSTM) algorithms were compared by multiple model evaluation metrics. The results showed that Bi LSTM models outperformed LSTM and GRU in predicting PAHs concentrations because it exhibited higher accuracy in terms of average R2ext (0.09) and errors (average MAPEext = 1.45). The DL interpretation based on SHAP values suggested that the main drivers of PAHs concentrations were PAHs emissions and sea ice area, with a more prominent contribution from meteorological conditions. The interpretable deep learning approach provides a potential shortcut for predicting time-delayed PAHs concentrations in the Arctic, promoting the management and control of global PAHs pollution.

Interpretable deep learning models for the inference and classification of LHC data

The Shower Deconstruction methodology is pivotal in distinguishing signal and background jets, leveraging the detailed information from perturbative parton showers. Rooted in the Neyman-Pearson lemma, this method is theoretically designed to differentiate between signal and background processes optimally in high-energy physics experiments. A key challenge, however, arises from the combinatorial growth associated with increasing jet constituents, which hampers its computational feasibility. We address this by demonstrating that the likelihood derived from comparing the most probable signal and background shower histories is equally effective for discrimination as the conventional approach of summing over all potential histories in top quark versus Quantum Chromodynamics (QCD) scenarios. We propose a novel approach by conceptualising the identification of the most probable shower history as a Markov Decision Process (MDP). Utilising a sophisticated modular point-transformer architecture, our method efficiently learns the optimal policy for this task. The developed neural agent excels in constructing the most likely shower history and demonstrates robust generalisation capabilities on unencountered test data. Remarkably, our approach mitigates the complexity inherent in the inference process, achieving a linear scaling relationship with the number of jet constituents. This offers a computationally viable and theoretically sound method for signal-background differentiation, paving the way for more effective data analysis in particle physics.

On the evaluation of deep learning interpretability methods for medical images under the scope of faithfulness

Background and Objective:Evaluating the interpretability of Deep Learning models is crucial for building trust and gaining insights into their decision-making processes. In this work, we employ class activation map based attribution methods in a setting where only High-Resolution Class Activation Mapping (HiResCAM) is known to produce faithful explanations. The objective is to evaluate the quality of the attribution maps using quantitative metrics and investigate whether faithfulness aligns with the metrics results. Methods:We fine-tune pre-trained deep learning architectures over four medical image datasets in order to calculate attribution maps. The maps are evaluated on a threefold metrics basis utilizing well-established evaluation scores. Results:Our experimental findings suggest that the Area Over Perturbation Curve (AOPC) and Max-Sensitivity scores favor the HiResCAM maps. On the other hand, the Heatmap Assisted Accuracy Score (HAAS) does not provide insights to our comparison as it evaluates almost all maps as inaccurate. To this purpose we further compare our calculated values against values obtained over a diverse group of models which are trained on non-medical benchmark datasets, to eventually achieve more responsive results. Conclusion:This study develops a series of experiments to discuss the connection between faithfulness and quantitative metrics over medical attribution maps. HiResCAM preserves the gradient effect on a pixel level ultimately producing high-resolution, informative and resilient mappings. In turn, this is depicted in the results of AOPC and Max-Sensitivity metrics, successfully identifying the faithful algorithm. In regards to HAAS, our experiments yield that it is sensitive over complex medical patterns, commonly characterized by strong color dependency and multiple attention areas.

Variational Attention-Based Interpretable Transformer Network for Rotary Machine Fault Diagnosis.

Deep learning technology provides a promising approach for rotary machine fault diagnosis (RMFD), where vibration signals are commonly utilized as input of a deep network model to reveal the internal state of machinery. However, most existing methods fail to mine association relationships within signals. Unlike deep neural networks, transformer networks are capable of capturing association relationships through the global self-attention mechanism to enhance feature representations from vibration signals. Despite this, transformer networks cannot explicitly establish the causal association between signal patterns and fault types, resulting in poor interpretability. To tackle these problems, an interpretable deep learning model named the variational attention-based transformer network (VATN) is proposed for RMFD. VATN is improved from transformer encoder to mine the association relationships within signals. To embed the prior knowledge of the fault type, which can be recognized based on several key features of vibration signals, a sparse constraint is designed for attention weights. Variational inference is employed to force attention weights to samples from Dirichlet distributions, and Laplace approximation is applied to realize reparameterization. Finally, two experimental studies conducted on bevel gear and bearing datasets demonstrate the effectiveness of VATN to other comparison methods, and the heat map of attention weights illustrates the causal association between fault types and signal patterns.

Deep learning the cis-regulatory code for gene expression in selected model plants

Elucidating the relationship between non-coding regulatory element sequences and gene expression is crucial for understanding gene regulation and genetic variation. We explored this link with the training of interpretable deep learning models predicting gene expression profiles from gene flanking regions of the plant species Arabidopsis thaliana, Solanum lycopersicum, Sorghum bicolor, and Zea mays. With over 80% accuracy, our models enabled predictive feature selection, highlighting e.g. the significant role of UTR regions in determining gene expression levels. The models demonstrated remarkable cross-species performance, effectively identifying both conserved and species-specific regulatory sequence features and their predictive power for gene expression. We illustrated the application of our approach by revealing causal links between genetic variation and gene expression changes across fourteen tomato genomes. Lastly, our models efficiently predicted genotype-specific expression of key functional gene groups, exemplified by underscoring known phenotypic and metabolic differences between Solanum lycopersicum and its wild, drought-resistant relative, Solanum pennellii.

Using an interpretable deep learning model for the prediction of riverine suspended sediment load.

The prediction of suspended sediment load (SSL) within riverine systems is critical to understanding the watershed's hydrology. Therefore, the novelty of our research is developing an interpretable (explainable) model based on deep learning (DL) and Shapley Additive ExPlanations (SHAP) interpretation technique for prediction of SSL in the riverine systems. This paper investigates the abilities of four DL models, including dense deep neural networks (DDNN), long short-term memory (LSTM), gated recurrent unit (GRU), and simple recurrent neural network (RNN) models for the prediction of daily SSL using river discharge and rainfall data at a daily time scale in the Taleghan River watershed, northwestern Tehran, Iran. The performance of models was evaluated by using several quantitative and graphical criteria. The effect of parameter settings on the performance of deep models on SSL prediction was also investigated. The optimal optimization algorithms, maximum iteration (MI), and batch size (BC) were obtained for modeling daily SSL, and structure of the model impact on prediction remarkably. The comparison of prediction accuracy of the models illustrated that DDNN (with R2 = 0.96, RMSE = 333.46) outperformed LSTM (R2 = 0.75, RMSE = 786.20), GRU (R2 = 0.73, RMSE = 825.67), and simple RNN (R2 = 0.78, RMSE = 741.45). Furthermore, the Taylor diagram confirmed that DDNN has the highest performance among other models. Interpretation techniques can address the black-box nature of models, and here, SHAP was applied to develop an interpretable DL model to interpret of DL model's output. The results of SHAP showed that river discharge has the strongest impact on the model's output in estimating SSL. Overall, we conclude that DL models have great potential in watersheds to predict SSL. Therefore, different interpretation techniques as tools to interpret DL model's output (DL model is as black-box model) are recommended in future research.

Development of an Interpretable Deep Learning Model for Pathological Tumor Response Assessment After Neoadjuvant Therapy.

Neoadjuvant therapy followed by surgery has become the standard of care for locally advanced esophageal squamous cell carcinoma (ESCC) and accurate pathological response assessment is critical to assess the therapeutic efficacy. However, it can be laborious and inconsistency between different observers may occur. Hence, we aim to develop an interpretable deep-learning model for efficient pathological response assessment following neoadjuvant therapy in ESCC. This retrospective study analyzed 337 ESCC resection specimens from 2020-2021 at the Pudong-Branch (Cohort 1) and 114 from 2021-2022 at the Puxi-Branch (External Cohort 2) of Fudan University Shanghai Cancer Center. Whole slide images (WSIs) from these two cohorts were generated using different scanning machines to test the ability of the model in handling color variations. Four pathologists independently assessed the pathological response. The senior pathologists annotated tumor beds and residual tumor percentages on WSIs to determine consensus labels. Furthermore, 1850 image patches were randomly extracted from Cohort 1 WSIs and binarily classified for tumor viability. A deep-learning model employing knowledge distillation was developed to automatically classify positive patches for each WSI and estimate the viable residual tumor percentages. Spatial heatmaps were output for model explanations and visualizations. The approach achieved high concordance with pathologist consensus, with an R^2 of 0.8437, a RAcc_0.1 of 0.7586, a RAcc_0.3 of 0.9885, which were comparable to two senior pathologists (R^2 of 0.9202/0.9619, RAcc_0.1 of 8506/0.9425, RAcc_0.3 of 1.000/1.000) and surpassing two junior pathologists (R^2 of 0.5592/0.5474, RAcc_0.1 of 0.5287/0.5287, RAcc_0.3 of 0.9080/0.9310). Visualizations enabled the localization of residual viable tumor to augment microscopic assessment. This work illustrates deep learning's potential for assisting pathological response assessment. Spatial heatmaps and patch examples provide intuitive explanations of model predictions, engendering clinical trust and adoption(Code and data will be available at https://github.com/WinnieLaugh/ESCC_Percentage once the paper has been conditionally accepted). Integrating interpretable computational pathology could help enhancethe efficiency and consistency of tumor response assessment and empower precise oncology treatment decisions.

Modified Adaptive CNN for Deep Learning based Histopathological Image Analysis for Cancer Diagnosis

This research on cancer diagnosis is motivated by the need for more accurate and efficient diagnostic tools in the field of oncology. Traditional methods of histopathological examination are time-consuming and can be subject to human error. The adoption of deep learning techniques, particularly Convolutional Neural Networks (CNNs), presents an opportunity to significantly improve the precision and speed of cancer diagnosis through automated analysis of tissue samples. This research addresses current challenges, including variability in samples and limited dataset diversity, with the goal of enhancing the reliability and applicability of the developed models. The ultimate aim is to contribute to the advancement of diagnostic methodologies, providing clinicians with more reliable tools for early and accurate cancer detection and ultimately improving patient outcomes and treatment strategies. Using a modified adaptive CNN algorithm, trained on extensive pathology image datasets, the approach enhances the precision and speed of cancer cell detection in tissue samples. By utilizing these artificial intelligence tools, the diagnostic process becomes not only more accurate but also significantly expeditious. Despite the promising strides, certain challenges persist. Variability in tissue samples, limited availability of diverse datasets, and interpretability of deep learning models pose hurdles to widespread adoption. To address these challenges, this research focuses on developing robust models that can generalize well across diverse datasets. Additionally, efforts are being made to create more comprehensive and diverse datasets to improve model training. Results show accuracy exceeding 97%, sensitivity and specificity at 97% and 95%, F1 score reaching 96%, and precision at 96%.