Interpretable Machine Learning Methods Research Articles

Analysis of runoff generation driving factors based on hydrological model and interpretable machine learning method

Study RegionXiaoqing River Basin, Shandong Province, China Study FocusIdentifying the driving factors of temporal and spatial variation in runoff is key to water resource management. The traditional machine learning model lacks transparency and interpretability, which affects the wide application of machine learning in the identification of influencing factors of hydrology. Interpretable machine learning method can improve the interpretability of machine learning model. The extreme gradient boosting (XGBoost) is established based on the data generated by the calibrated Soil Water Assessment Tool (SWAT), and the XGBoost is interpreted using the Shapely additive explanations (SHAP) method to identify the impact of driving factors on runoff generation. New hydrological insights for the regionThe results show that XGBoost can simulate the simulation ability of SWAT, and SHAP can identify the factors affecting runoff generation by interpreting XGBoost. It was found that climatic features have different effects on runoff in different sub-basins, and rainfall at high elevations (or slope) has stronger effects on runoff than that at low elevations. There is an obvious threshold effect of land use combination (or slope) on the generation of runoff, and this threshold effect is driven by high precipitation. The results of this study can provide a new method for factor analysis of runoff.

Charge-dependent Fermi level of graphene oxide nanoflakes from machine learning

Although the energy of the Fermi level is of critical importance to designing electrically conductive materials, heterostructures and devices, the relationship between the Fermi energy and complex structure of graphene oxide has been difficult to predict due to competing dependencies on oxygen concentration and distribution, defects and charge. In this study we have used a data set of over 60,000 unique graphene oxide nanostructures and interpretable machine learning methods to show that the principal determinant is the ionic charge, which is in itself structure-independent. From this we define three separate, highly accurate, charge-dependent structure/property relationships and show that the Fermi energy can be predicted based on the ether concentration, hydrogen passivation or size, for the neutral, anionic and cationic cases, respectively. These important features can inform experimental design, and are remarkably insensitive to minor structural variations that are difficult to control in the lab.

Use of interpretable machine learning to identify the factors influencing the nonlinear linkage between land use and river water quality in the Chesapeake Bay watershed

Understanding the relationship between land use and water quality is essential for effective watershed management. However, it remains challenging to identify such a relationship owing to its nonlinearity. We developed an interpretable machine learning method that integrated the random forest regression (RFR) model with the Shapley Additive exPlanations (SHAP) method to explore the relationship between water quality and land use in the Potomac River Basin (PRB), the second largest tributary entering Chesapeake Bay from 2006 to 2019. The water quality of the 26 sub-watersheds, classified into five types (natural, forested, agricultural, mixed, and urbanized), was investigated using statistical methods and scenario analysis. The results showed that the models employed were effective in predicting the water quality. The mean absolute error (MAE), root mean square error (RMSE), percent bias (PBIAS), R2 coefficient of determination (R2), and Kling-Gupta efficiency (KGE) were 0.011–0.159 mg/L, 0.019–0.219 mg/L, −0.14–0.64%, 0.79–0.99, and 0.69–0.98, respectively, during the training period, which were 0.010–0.201 mg/L, 0.017–0.292 mg/L, −1.87–0.41%, 0.82–0.99, and 0.80–0.97, respectively, during the testing period. The threshold effects of land use patterns were obvious for water quality indicators with high concentrations (i.e., TN and NH4+-N). In contrast, the water quality at low concentrations (i.e., TP and NO3–-N) may be more sensitive to wetland or barren land with changing climate. Agricultural activities and urbanization could be the dominant factors determining nutrient export to the PRB. Meanwhile, the typical ‘sink’ for the nutrient such as wetland may change into the ‘source’ for different nutrient. This study provides an in-depth understanding of how riverine nutrient export responds to the land use gradient in the Chesapeake Bay watershed.

An associative knowledge network model for interpretable semantic representation of noun context

Uninterpretability has become the biggest obstacle to the wider application of deep neural network, especially in most human–machine interaction scenes. Inspired by the powerful associative computing ability of human brain neural system, a novel interpretable semantic representation model of noun context, associative knowledge network model, is proposed. The proposed network structure is composed of only pure associative relationships without relation label and is dynamically generated by analysing neighbour relationships between noun words in text, in which incremental updating and reduction reconstruction strategies can be naturally introduced. Furthermore, a novel interpretable method is designed for the practical problem of checking the semantic coherence of noun context. In proposed method, the associative knowledge network learned from the text corpus is first regarded as a background knowledge network, and then the multilevel contextual associative coupling degree features of noun words in given detection document are computed. Finally, contextual coherence detection and the location of those inconsistent noun words can be realized by using an interpretable classification method such as decision tree. Our sufficient experimental results show that above proposed method can obtain excellent performance and completely reach or even partially exceed the performance obtained by the latest deep neural network methods especially in F1 score metric. In addition, the natural interpretability and incremental learning ability of our proposed method should be extremely valuable than deep neural network methods. So, this study provides a very enlightening idea for developing interpretable machine learning methods, especially for the tasks of text semantic representation and writing error detection.

Demystifying the Black Box: The Importance of Interpretability of Predictive Models in Neurocritical Care

Neurocritical care patients are a complex patient population, and to aid clinical decision-making, many models and scoring systems have previously been developed. More recently, techniques from the field of machine learning have been applied to neurocritical care patient data to develop models with high levels of predictive accuracy. However, although these recent models appear clinically promising, their interpretability has often not been considered and they tend to be black box models, making it extremely difficult to understand how the model came to its conclusion. Interpretable machine learning methods have the potential to provide the means to overcome some of these issues but are largely unexplored within the neurocritical care domain. This article examines existing models used in neurocritical care from the perspective of interpretability. Further, the use of interpretable machine learning will be explored, in particular the potential benefits and drawbacks that the techniques may have when applied to neurocritical care data. Finding a solution to the lack of model explanation, transparency, and accountability is important because these issues have the potential to contribute to model trust and clinical acceptance, and, increasingly, regulation is stipulating a right to explanation for decisions made by models and algorithms. To ensure that the prospective gains from sophisticated predictive models to neurocritical care provision can be realized, it is imperative that interpretability of these models is fully considered.

Improved Understanding of How Catchment Properties Control Hydrological Partitioning Through Machine Learning

AbstractLong‐term hydrological partitioning of catchments can be well described by the Budyko framework with a parameter (e.g., Fu's equations with parameter ω). The Budyko framework considers aridity index as the dominant control on hydrological partitioning, while the parameter represents integrated influences of catchment properties. Our understanding regarding the controls of catchment properties on the parameter is still limited. In this study, two machine learning methods, that is, boosted regression tree (BRT) and CUBIST, were used to model ω. Interpretable machine learning methods were adopted for better physical understanding including feature importance, accumulated local effects (ALE), and local interpretable model‐agnostic explanations. Among the 15 properties of 443 Australian catchments, analysis of feature importance showed that root zone storage capacity (SR), vapor pressure, vegetation coverage (M), precipitation depth, climate seasonality and asynchrony index (SAI), and water use efficiency (WUE) were the six primary control factors on ω. ALE showed that ω varied nonlinearly with all factors, and varied non‐monotonically with M, SAI, and WUE. LIME showed that the importance of the six dominant factors on ω varied between regions. CUBIST was further used to build regionally varying relationships between ω and the primary factors. Continental scale ω and evapotranspiration were further mapped across Australia based on the most robust BRT‐trained parameterization scheme with a resolution of 0.05°. Instead of using the machine learning method as a black box, we employed interpretability approaches to identify the controls. Our findings not only improved the capability of the Budyko method for hydrological partitioning across Australia, but also demonstrated that the controls of catchment properties on hydrological partitioning vary in different regions.

Non-laboratory-based risk assessment model for case detection of diabetes mellitus and pre-diabetes in primary care.

IntroductionMore than half of diabetes mellitus (DM) and pre‐diabetes (pre‐DM) cases remain undiagnosed, while existing risk assessment models are limited by focusing on diabetes mellitus only (omitting pre‐DM) and often lack lifestyle factors such as sleep. This study aimed to develop a non‐laboratory risk assessment model to detect undiagnosed diabetes mellitus and pre‐diabetes mellitus in Chinese adults.MethodsBased on a population‐representative dataset, 1,857 participants aged 18–84 years without self‐reported diabetes mellitus, pre‐diabetes mellitus, and other major chronic diseases were included. The outcome was defined as a newly detected diabetes mellitus or pre‐diabetes by a blood test. The risk models were developed using logistic regression (LR) and interpretable machine learning (ML) methods. Models were validated using area under the receiver‐operating characteristic curve (AUC‐ROC), precision‐recall curve (AUC‐PR), and calibration plots. Two existing diabetes mellitus risk models were included for comparison.ResultsThe prevalence of newly diagnosed diabetes mellitus and pre‐diabetes mellitus was 15.08%. In addition to known risk factors (age, BMI, WHR, SBP, waist circumference, and smoking status), we found that sleep duration, and vigorous recreational activity time were also significant risk factors of diabetes mellitus and pre‐diabetes mellitus. Both LR (AUC‐ROC = 0.812, AUC‐PR = 0.448) and ML models (AUC‐ROC = 0.822, AUC‐PR = 0.496) performed well in the validation sample with the ML model showing better discrimination and calibration. The performance of the models was better than the two existing models.ConclusionsSleep duration and vigorous recreational activity time are modifiable risk factors of diabetes mellitus and pre‐diabetes in Chinese adults. Non‐laboratory‐based risk assessment models that incorporate these lifestyle factors can enhance case detection of diabetes mellitus and pre‐diabetes.

Identification of new marker genes from plant single-cell RNA-seq data using interpretable machine learning methods.

Summary An essential step in the analysis of single‐cell RNA sequencing data is to classify cells into specific cell types using marker genes. In this study, we have developed a machine learning pipeline called single‐cell predictive marker (SPmarker) to identify novel cell‐type marker genes in the Arabidopsis root.Unlike traditional approaches, our method uses interpretable machine learning models to select marker genes. We have demonstrated that our method can: assign cell types based on cells that were labelled using published methods; project cell types identified by trajectory analysis from one data set to other data sets; and assign cell types based on internal GFP markers.Using SPmarker, we have identified hundreds of new marker genes that were not identified before. As compared to known marker genes, the new marker genes have more orthologous genes identifiable in the corresponding rice single‐cell clusters. The new root hair marker genes also include 172 genes with orthologs expressed in root hair cells in five non‐Arabidopsis species, which expands the number of marker genes for this cell type by 35–154%.Our results represent a new approach to identifying cell‐type marker genes from scRNA‐seq data and pave the way for cross‐species mapping of scRNA‐seq data in plants.

Prediction of Online Psychological Help-Seeking Behavior During the COVID-19 Pandemic: An Interpretable Machine Learning Method.

Online mental health service (OMHS) has been named as the best psychological assistance measure during the COVID-19 pandemic. An interpretable, accurate, and early prediction for the demand of OMHS is crucial to local governments and organizations which need to allocate and make the decision in mental health resources. The present study aimed to investigate the influence of the COVID-19 pandemic on the online psychological help-seeking (OPHS) behavior in the OMHS, then propose a machine learning model to predict and interpret the OPHS number in advance. The data was crawled from two Chinese OMHS platforms. Linguistic inquiry and word count (LIWC), neural embedding-based topic modeling, and time series analysis were utilized to build time series feature sets with lagging one, three, seven, and 14 days. Correlation analysis was used to examine the impact of COVID-19 on OPHS behaviors across different OMHS platforms. Machine learning algorithms and Shapley additive explanation (SHAP) were used to build the prediction. The result showed that the massive growth of OPHS behavior during the COVID-19 pandemic was a common phenomenon. The predictive model based on random forest (RF) and feature sets containing temporal features of the OPHS number, mental health topics, LIWC, and COVID-19 cases achieved the best performance. Temporal features of the OPHS number showed the biggest positive and negative predictive power. The topic features had incremental effects on performance of the prediction across different lag days and were more suitable for OPHS prediction compared to the LIWC features. The interpretable model showed that the increase in the OPHS behaviors was impacted by the cumulative confirmed cases and cumulative deaths, while it was not sensitive in the new confirmed cases or new deaths. The present study was the first to predict the demand for OMHS using machine learning during the COVID-19 pandemic. This study suggests an interpretable machine learning method that can facilitate quick, early, and interpretable prediction of the OPHS behavior and to support the operational decision-making; it also demonstrated the power of utilizing the OMHS platforms as an always-on data source to obtain a high-resolution timeline and real-time prediction of the psychological response of the online public.

Abstract WP230: Interpretable Machine Learning Models For Predicting 30-Day All-Cause Readmission Following Carotid Endarterectomy Among Acute Ischemic Stroke Cases: A NSQIP Study (2014 - 2017)

Background: Acute ischemic stroke (AIS) patients show a good prognosis with Carotid endarterectomy (CEA). However, lowering short-term readmissions after CEA for AIS cases remains a significant challenge. Several machine learning (ML) models predicting readmissions fail to explain the predictors. Hence the utility of such ML models in clinical settings is limited. This study used interpretable machine learning (ML) methods to identify the predictors associated with 30-day readmission. Methods: We utilized the National Surgical Quality Improvement Program registry (2014-2017) for this study. Patients aged >18 years and who underwent CEA for AIS were included. ICD-9, ICD-10, and CPT codes were used to identify AIS and CEA cases. Decision Trees and Random Forest classification techniques were utilized to identify predictors of 30-day readmission. Results: A total of 22,373 AIS patients underwent CEA during the study period. The mean (SD) age of the patients was 70.7 (9.4) years, and the majority (61%) were males. About 80% were Non-Hispanic White, followed by non-Hispanic Black (4.6%). About 7% of AIS patients who underwent CEA had 30-day readmission. Random Forest classification and Decision Tree were able to provide clinically relevant predictors and cut-off values. For example, one of the top predictors was pre-operative Hematocrit, and its cut-off value of ≤33% with Diabetes showed a higher risk for readmission. Conclusion: Our study showed that interpretable ML models could be helpful for clinicians to stratify patients based on their pre-operative risk for a 30-day readmission and could help optimize management strategies for improving patient outcomes. Incorporating these cut-off values in EMR could help clinical decision-making and plan interventions to reduce readmissions by managing risk factors. This would improve the quality of care and save high healthcare costs.

Chain Graph Explanation of Neural Network Based on Feature-Level Class Confusion

Despite increasing interest in developing interpretable machine learning methods, most recent studies have provided explanations only for single instances, require additional datasets, and are sensitive to hyperparameters. This paper proposes a confusion graph that reveals model weaknesses by constructing a confusion dictionary. Unlike other methods, which focus on the performance variation caused by single-neuron suppression, it defines the role of each neuron in two different perspectives: ‘correction’ and ‘violation’. Furthermore, our method can identify the class relationships in similar positions at the feature level, which can suggest improvements to the model. Finally, the proposed graph construction is model-agnostic and does not require additional data or tedious hyperparameter tuning. Experimental results show that the information loss from omitting the channels guided by the proposed graph can result in huge performance degradation, from 91% to 33%, while the proposed graph only retains 1% of total neurons.

MultiCapsNet: A General Framework for Data Integration and Interpretable Classification.

The latest progresses of experimental biology have generated a large number of data with different formats and lengths. Deep learning is an ideal tool to deal with complex datasets, but its inherent “black box” nature needs more interpretability. At the same time, traditional interpretable machine learning methods, such as linear regression or random forest, could only deal with numerical features instead of modular features often encountered in the biological field. Here, we present MultiCapsNet (https://github.com/wanglf19/MultiCapsNet), a new deep learning model built on CapsNet and scCapsNet, which possesses the merits such as easy data integration and high model interpretability. To demonstrate the ability of this model as an interpretable classifier to deal with modular inputs, we test MultiCapsNet on three datasets with different data type and application scenarios. Firstly, on the labeled variant call dataset, MultiCapsNet shows a similar classification performance with neural network model, and provides importance scores for data sources directly without an extra importance determination step required by the neural network model. The importance scores generated by these two models are highly correlated. Secondly, on single cell RNA sequence (scRNA-seq) dataset, MultiCapsNet integrates information about protein-protein interaction (PPI), and protein-DNA interaction (PDI). The classification accuracy of MultiCapsNet is comparable to the neural network and random forest model. Meanwhile, MultiCapsNet reveals how each transcription factor (TF) or PPI cluster node contributes to classification of cell type. Thirdly, we made a comparison between MultiCapsNet and SCENIC. The results show several cell type relevant TFs identified by both methods, further proving the validity and interpretability of the MultiCapsNet.

Towards a metagenomics machine learning interpretable model for understanding the transition from adenoma to colorectal cancer

Gut microbiome is gaining interest because of its links with several diseases, including colorectal cancer (CRC), as well as the possibility of being used to obtain non-intrusive predictive disease biomarkers. Here we performed a meta-analysis of 1042 fecal metagenomic samples from seven publicly available studies. We used an interpretable machine learning approach based on functional profiles, instead of the conventional taxonomic profiles, to produce a highly accurate predictor of CRC with better precision than those of previous proposals. Moreover, this approach is also able to discriminate samples with adenoma, which makes this approach very promising for CRC prevention by detecting early stages in which intervention is easier and more effective. In addition, interpretable machine learning methods allow extracting features relevant for the classification, which reveals basic molecular mechanisms accounting for the changes undergone by the microbiome functional landscape in the transition from healthy gut to adenoma and CRC conditions. Functional profiles have demonstrated superior accuracy in predicting CRC and adenoma conditions than taxonomic profiles and additionally, in a context of explainable machine learning, provide useful hints on the molecular mechanisms operating in the microbiota behind these conditions.

Explainable K-Means Clustering for Occupancy Estimation

Explainable AI (Artificial Intelligence) has gained significant popularity in recent years. It provides reliable and helpful insights about the model and its output predictions. Even though Explainable AI is an important topic thanks to its interpretability, there have been only very few works that are conducted under the unsupervised learning task. This research aims to apply an interpretable unsupervised machine learning method to the task of Occupancy Estimation (OE). Estimating the number of occupants in buildings has been proven to be helpful to several applications such as energy management. Explainable AI increases the credibility of the applications based on OE by giving the user an interpretation of the obtained OE predictions using simple If-Then rules. In all previous research works on clustering methods either results were so difficult to interpret or data were hard to label. To avert these problems and enhance the understanding of the clustering decision, we propose a small binary decision tree to divide the dataset into k clusters so that the interpretability of the obtained k clusters will be simple to understand. It is known that with an increase in the number of leaves of the decision tree, the accuracy of the clusters increases, the cost of clustering increases, and the interpretability of the obtained clusters will be more difficult to deal with. In this work, we applied and tested an unsupervised machine learning model known as the explainable K-means clustering (ExKMC) algorithm for occupancy estimation. ExKMC by default creates a small tree with k leaves that partitions the data into k clusters, and it also outputs a new tree with k' leaves where k' ≥k that provides explainable clusters. The considered method makes a simple trade-off between the accuracy of prediction and the interpretability of the clustering decisions.

Research on Personal Credit Risk Assessment Based on Combination Weight and Shap Interpretable Machine Learning

With the continuous promotion of e-commerce platform installment payment and P2P credit platform, personal credit risk assessment is becoming more and more important for e-commerce and users Taking the data of Tianchi competition platform as a sample, this paper constructs an index system for personal credit risk assessment, and calculates the combined weight of each index through the critical method and entropy weight method, and weights it to obtain three factors: basic information, credit information and lending behavior information. Based on the quantified three indicators and the unbalanced sample algorithm, XGBoost and lightGBM are used to predict credit risk, it is found that the performance of these two methods is basically the same. This paper uses the SHAP-values interpretable machine learning method to explain the importance of these three factors. The empirical results show that the accuracy of XGBoost and lightGBM is higher than 80%, and the order of the importance of the three factors is: "basic information" is higher than "lending behavior information" and higher than "credit information". Finally, this paper puts forward relevant suggestions for the stable development of enterprise risk control and credit industry.

Unilateral Alignment: An interpretable machine learning method for geophysical logs calibration

Most of the existing machine learning studies in logs interpretation do not consider the data distribution discrepancy issue, so the trained model cannot well generalize to the unseen data without calibrating the logs. In this paper, we formulated the geophysical logs calibration problem and give its statistical explanation, and then exhibited an interpretable machine learning method, i.e., Unilateral Alignment, which could align the logs from one well to another without losing the physical meanings. The involved UA method is an unsupervised feature domain adaptation method, so it does not rely on any labels from cores. The experiments in 3 wells and 6 tasks showed the effectiveness and interpretability from multiple views.

Discover immunotherapy biomarkers from single-cell cytometry data

Currently, identifying novel biomarkers remains a crucial need for cancer immunotherapy. By leveraging single-cell cytometry data, Greene et al. developed an interpretable machine learning method, FAUST, to discover cell populations associated with clinical outcomes.

An Interaction-Based Convolutional Neural Network (ICNN) Toward a Better Understanding of COVID-19 X-ray Images

The field of explainable artificial intelligence (XAI) aims to build explainable and interpretable machine learning (or deep learning) methods without sacrificing prediction performance. Convolutional neural networks (CNNs) have been successful in making predictions, especially in image classification. These popular and well-documented successes use extremely deep CNNs such as VGG16, DenseNet121, and Xception. However, these well-known deep learning models use tens of millions of parameters based on a large number of pretrained filters that have been repurposed from previous data sets. Among these identified filters, a large portion contain no information yet remain as input features. Thus far, there is no effective method to omit these noisy features from a data set, and their existence negatively impacts prediction performance. In this paper, a novel interaction-based convolutional neural network (ICNN) is introduced that does not make assumptions about the relevance of local information. Instead, a model-free influence score (I-score) is proposed to directly extract the influential information from images to form important variable modules. This innovative technique replaces all pretrained filters found by trial-and-error with explainable, influential, and predictive variable sets (modules) determined by the I-score. In other words, future researchers need not rely on pretrained filters; the suggested algorithm identifies only the variables or pixels with high I-score values that are extremely predictive and important. The proposed method and algorithm were tested on real-world data set and a state-of-the-art prediction performance of 99.8% was achieved without sacrificing the explanatory power of the model. This proposed design can efficiently screen patients infected by COVID-19 before human diagnosis and can be a benchmark for addressing future XAI problems in large-scale data sets.

P57.04 Predicting Treatment Response to 1st- line Pembrolizumab in Advanced Non-Small Cell Lung Cancer (NSCLC) Patients with High PDL1 Expression

Combination of pembrolizumab and chemotherapy has shown improved survival in advanced NSCLC compared to chemotherapy alone. However, the overall response rate has only increased from 45% to 60% by adding chemotherapy to pembrolizumab alone for patients with high PDL1 expression (PD-L1 ≥50%). Currently no biomarker or clinicopathological features can be used to identify the subgroup with tumors PD-L1 ≥ 50% who would respond well to pembrolizumab alone in the 1st-line setting. We hypothesize a machine learning model trained on pre-treatment CT image radiomic features can be used as a digital biomarker to select the aforementioned subgroup, thereby reducing chemotherapy related toxicity and mitigating health care expenditure. We retrospectively identified stage III/IV NSCLC patients who received 1st-line single pembrolizumab at British Columbia Cancer (Vancouver, Canada), with PDL1 ≥50% and all had baseline staging CT within 6 weeks prior to starting immunotherapy and the 1st follow-up (FU) CT within 12 weeks after treatment initiation. The baseline and 1st follow-up CTs were reviewed by two oncological radiologists to identified the lung tumors on the baseline CT images and assess tumor response (i.e., partial response, PR vs progressive disease, PD) using RECIST 1.1 criteria. Utilizing an in-house CT Otsu based thresholding segmentation program and radiomic feature extraction pipeline, we identified potentially discriminating features from CT lung tumor images. We performed sequential forward feature selection to identify the 10 most highly classifying features from discrete shape, texture, and intensity radiomic features generated for 3 distinct tumor segmentation masks (lesion core, core with perimeter transition pixels, and the ring of parenchymal tissue surrounding the lesion). In this study, we leveraged an 8-Xfold validation Linear Discriminate Analysis (LDA), a simple machine learning method trained on these 10 features, to discriminate tumor response to 1st-line pembrolizumab as evaluated at FU CT scans. Sixty-eight patients were included (44% male, 73±6 yrs, 52ES/12CS/4NS; pack years: 43±18; ECOG range: 0-3). Twenty-eight showed response to the 1st-line pembrolizumab and forty had progressive disease. We identified eighty-four lung tumors in these patients which we extracted radiomic features from the five central slices, providing us with a pseudo-volumetric CT image training and test set (N=380). ROC analysis of the LDA model resulted in an accuracy of 79.4% [SN: 0.882, SP: 0.684] (AUC: 0.79) for our patient dataset. Given our class label imbalance, we performed precision-recall (PPV vs Sensitivity) analysis to identify sources of class bias (AUC vs No-skill AUC: 0.80 vs 0.59). Our data showed that a rudimentary and interpretable machine learning method can lead to appropriate treatment paths for advanced stage lung cancers. A combination of all mask features performed well on this task with a majority of the features selected being texture and intensity features from the core plus edge transitionary boundary pixels. Future analysis will be conducted using other machine learning methods to validate these findings, while keeping the LDA as the primary analysis method.

Explainable Artificial Intelligence for Human Decision Support System in the Medical Domain

In this paper, we present the potential of Explainable Artificial Intelligence methods for decision support in medical image analysis scenarios. Using three types of explainable methods applied to the same medical image data set, we aimed to improve the comprehensibility of the decisions provided by the Convolutional Neural Network (CNN). In vivo gastral images obtained by a video capsule endoscopy (VCE) were the subject of visual explanations, with the goal of increasing health professionals’ trust in black-box predictions. We implemented two post hoc interpretable machine learning methods, called Local Interpretable Model-Agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP), and an alternative explanation approach, the Contextual Importance and Utility (CIU) method. The produced explanations were assessed by human evaluation. We conducted three user studies based on explanations provided by LIME, SHAP and CIU. Users from different non-medical backgrounds carried out a series of tests in a web-based survey setting and stated their experience and understanding of the given explanations. Three user groups (n = 20, 20, 20) with three distinct forms of explanations were quantitatively analyzed. We found that, as hypothesized, the CIU-explainable method performed better than both LIME and SHAP methods in terms of improving support for human decision-making and being more transparent and thus understandable to users. Additionally, CIU outperformed LIME and SHAP by generating explanations more rapidly. Our findings suggest that there are notable differences in human decision-making between various explanation support settings. In line with that, we present three potential explainable methods that, with future improvements in implementation, can be generalized to different medical data sets and can provide effective decision support to medical experts.