Interpretable Machine Learning Models Research Articles

Wildfire risk exploration: leveraging SHAP and TabNet for precise factor analysis

BackgroundUnderstanding the intricacies of wildfire impact across diverse geographical landscapes necessitates a nuanced comprehension of fire dynamics and areas of vulnerability, particularly in regions prone to high wildfire risks. Machine learning (ML) stands as a formidable ally in addressing the complexities associated with predicting and mapping these risks, offering advanced analytical capabilities. Nevertheless, the reliability of such ML approaches is heavily contingent on the integrity of data and the robustness of training protocols. The scientific community has raised concerns about the transparency and interpretability of ML models in the context of wildfire management, recognizing the need for these models to be both accurate and understandable. The often-opaque nature of complex ML algorithms can obscure the rationale behind their outputs, making it imperative to prioritize clarity and interpretability to ensure that model predictions are not only precise but also actionable. Furthermore, a thorough evaluation of model performance must account for multiple critical factors to ensure the utility and dependability of the results in practical wildfire suppression and management strategies.ResultsThis study unveils a sophisticated spatial deep learning framework grounded in TabNet technology, tailored specifically for delineating areas susceptible to wildfires. To elucidate the predictive interplay between the model’s outputs and the contributing variables across a spectrum of inputs, we embark on an exhaustive analysis using SHapley Additive exPlanations (SHAP). This approach affords a granular understanding of how individual features sway the model’s predictions. Furthermore, the robustness of the predictive model is rigorously validated through 5-fold cross-validation techniques, ensuring the dependability of the findings. The research meticulously investigates the spatial heterogeneity of wildfire susceptibility within the designated study locale, unearthing pivotal insights into the nuanced fabric of fire risk that is distinctly local in nature.ConclusionUtilizing SHapley Additive exPlanations (SHAP) visualizations, this research meticulously identifies key variables, quantifies their importance, and demystifies the decision-making mechanics of the model. Critical factors, including temperature, elevation, the Normalized Difference Vegetation Index (NDVI), aspect, and wind speed, are discerned to have significant sway over the predictions of wildfire susceptibility. The findings of this study accentuate the criticality of transparency in modeling, which facilitates a deeper understanding of wildfire risk factors. By shedding light on the significant predictors within the models, this work enhances our ability to interpret complex predictive models and drives forward the field of wildfire risk management, ultimately contributing to the development of more effective prevention and mitigation strategies.

Coupling of Solidification and Heat Transfer Simulations with Interpretable Machine Learning Algorithms to Predict Transverse Cracks in Continuous Casting of Steel

The formation of defects such as cracks in continuous casting deteriorates the quality of cast products and efficiency of steelmaking. To evaluate the risks and identify the root causes of defect formation, phenomenological quality criteria computed with a solidification and microstructure model known as InterDendritic Solidification (IDS) have previously been applied. This approach is computationally efficient and provides a fundamental perspective to defect formation in continuous casting. The aim of this work is to study the capabilities of these criteria as features in predicting transverse cracking with interpretable machine learning models. IDS is coupled with a heat transfer model known as Tempsimu to simulate the continuous casting process. Measured compositions are utilized in the simulations and defects reported at a steelmaking plant are used as labels in classification. Logistic regression, decision tree, and Gaussian Naïve Bayes classifiers are developed to predict transverse cracking in peritectic C–Mn, low‐carbon B–Ti microalloyed, and peritectic Nb‐microalloyed steels. The corresponding balanced accuracies of the best classifiers from cross‐validation are 92%, 94.6%, and 82.8%. Due to the good performance and the interpretability of the developed classifiers, the fundamental causes of transverse cracking and possibilities of avoiding it by changes in the compositions are identified.

Interpretable machine learning model for prediction of overall survival in laryngeal cancer

Background: The mortality rates of laryngeal squamous cell carcinoma cancer (LSCC) have not significantly decreased in the last decades. Objectives: We primarily aimed to compare the predictive performance of DeepTables with the state-of-the-art machine learning (ML) algorithms (Voting ensemble, Stack ensemble, and XGBoost) to stratify patients with LSCC into chance of overall survival (OS). In addition, we complemented the developed model by providing interpretability using both global and local model-agnostic techniques. Methods: A total of 2792 patients in the Surveillance, Epidemiology, and End Results (SEER) database diagnosed with LSCC were reviewed. The global model-agnostic interpretability was examined using SHapley Additive exPlanations (SHAP) technique. Likewise, individual interpretation of the prediction was made using Local Interpretable Model Agnostic Explanations (LIME). Results: The state-of-the-art ML ensemble algorithms outperformed DeepTables. Specifically, the examined ensemble algorithms showed comparable weighted area under receiving curve of 76.9, 76.8, and 76.1 with an accuracy of 71.2%, 70.2%, and 71.8%, respectively. The global methods of interpretability (SHAP) demonstrated that the age of the patient at diagnosis, N-stage, T-stage, tumor grade, and marital status are among the prominent parameters. Conclusions: A ML model for OS prediction may serve as an ancillary tool for treatment planning of LSCC patients.

Constructing transferable and interpretable machine learning models for black carbon concentrations

Black carbon (BC) has received increasing attention from researchers due to its adverse health effects. However, in-situ BC measurements are often not included as a regulated variable in air quality monitoring networks. Machine learning (ML) models have been studied extensively to serve as virtual sensors to complement the reference instruments. This study evaluates and compares three white-box (WB) and four black-box (BB) ML models to estimate BC concentrations, with the focus to show their transferability and interpretability. We train the models with the long-term air pollutant and weather measurements in Barcelona urban background site, and test them in other European urban and traffic sites. Despite the difference in geographical locations and measurement sites, BC correlates the strongest with particle number concentration of accumulation mode (PNacc, r = 0.73–0.85) and nitrogen dioxide (NO2, r = 0.68–0.85) and the weakest with meteorological parameters. Due to its similarity of correlation behaviour, the ML models trained in Barcelona performs prominently at the traffic site in Helsinki (R2 = 0.80–0.86; mean absolute error MAE = 3.90–4.73 %) and at the urban background site in Dresden (R2 = 0.79–0.84; MAE = 4.23–4.82 %). WB models appear to explain less variability of BC than BB models, long short-term memory (LSTM) model of which outperforms the rest of the models. In terms of interpretability, we adopt several methods for individual model to quantify and normalize the relative importance of each input feature. The overall static relative importance commonly used for WB models demonstrate varying results from the dynamic values utilized to show local contribution used for BB models. PNacc and NO2 on average have the strongest absolute static contribution; however, they simultaneously impact the estimation positively and negatively at different sites. This comprehensive analysis demonstrates that the possibility of these interpretable air pollutant ML models to be transfered across space and time.

Data‐Driven Decoupling Structural Feature Correlation for Harnessing Anionic Capacity in Na Layered Oxide

AbstractA data‐driven understanding of the reaction mechanism of the O3‐type Na[Li1/3Mn2/3]O2‐layered cathode model is systematically performed to decouple the complex correlations of covariate scale‐dependent variables in diagnosing and solving structural problems for facilitating nonhysteretic and reversible (nHR) oxygen capacities using interpretable machine learning (ML) assisted by density functional theory. A large dataset of vacancy formation energies depending on the desodiation mode for the oxide is investigated in detail, and it provides two numerical principles: i) linearizing the energy landscape and ii) steepening its slope to reach the ideal reaction. The heatmaps comprising Pearson coefficient correlation values are broken down into two‐scale components: i) macroscopic and ii) local structure features. Deriving the overall mitigation of scale‐dependent covariate variables in negative correlation potentially leads to nHR anionic redox upon (dis)charging. Containing the scale‐dependent features, the interpretable ML model based on a gradient‐boosting machine predicts each formation energy well. With data‐driven comprehension, honeycomb‐ and turtle‐type superstructures (TS) have been suggested depending on the thermodynamic (un)favorable pathways during desodiation from a local structural perspective, and the dangling O2– in the TS is a critical origin leading to the formation of O2 molecules trapped in the bulk.

Predicting systemic financial risk with interpretable machine learning

Predicting systemic financial risk is essential for understanding the financial system's stability and early warning of financial crises. In this research, we use the financial stress index to measure systemic financial risk. We construct the stress index for five financial submarkets and composite stress index, employ the Markov regime switching model to identify the systemic financial risk stress state. On this basis, we use interpretable machine learning models to forecast systemic financial risk, analyze and compare the results of the intrinsic interpretable machine learning models and the post-hoc explainable methods. The results indicate that systemic financial risk can be effectively predicted using both the submarket stress index and the feature variables, with the submarket stress index as the independent variable providing relatively higher accuracy. There is a linearly positive relationship between the stress index of each submarket and systemic financial risk, with financial stress in the stock and money markets having the greatest impact on systemic financial risk. For each feature variable, stock–bond correlation coefficient, stock valuation risk, the maximum cumulative loss of the SSE Composite Index (SSE CMAX), and loan-deposit ratio have strong predictive power. Our research can provide reference for government to construct prediction model and indicator monitoring platform of systemic financial crisis.

Transparency in AI Decision Making: A Survey of Explainable AI Methods and Applications

Artificial Intelligence (AI) systems have become pervasive in numerous facets of modern life, wielding considerable influence in critical decision-making realms such as healthcare, finance, criminal justice, and beyond. Yet, the inherent opacity of many AI models presents significant hurdles concerning trust, accountability, and fairness. To address these challenges, Explainable AI (XAI) has emerged as a pivotal area of research, striving to augment the transparency and interpretability of AI systems. This survey paper serves as a comprehensive exploration of the state-of-the-art in XAI methods and their practical applications. We delve into a spectrum of techniques, spanning from model-agnostic approaches to interpretable machine learning models, meticulously scrutinizing their respective strengths, limitations, and real-world implications. The landscape of XAI is rich and varied, with diverse methodologies tailored to address different facets of interpretability. Model-agnostic approaches offer versatility by providing insights into model behavior across various AI architectures. In contrast, interpretable machine learning models prioritize transparency by design, offering inherent understandability at the expense of some predictive performance. Layer-wise Relevance Propagation (LRP) and attention mechanisms delve into the inner workings of neural networks, shedding light on feature importance and decision processes. Additionally, counterfactual explanations open avenues for exploring what-if scenarios, elucidating the causal relationships between input features and model outcomes. In tandem with methodological exploration, this survey scrutinizes the deployment and impact of XAI across multifarious domains. Successful case studies showcase the practical utility of transparent AI in healthcare diagnostics, financial risk assessment, criminal justice systems, and more. By elucidating these use cases, we illuminate the transformative potential of XAI in enhancing decision-making processes while fostering accountability and fairness. Nevertheless, the journey towards fully transparent AI systems is fraught with challenges and opportunities. As we traverse the current landscape of XAI, we identify pressing areas for further research and development. These include refining interpretability metrics, addressing the scalability of XAI techniques to complex models, and navigating the ethical dimensions of transparency in AI decision-making.Through this survey, we endeavor to cultivate a deeper understanding of transparency in AI decision-making, empowering stakeholders to navigate the intricate interplay between accuracy, interpretability, and ethical considerations. By fostering interdisciplinary dialogue and inspiring collaborative innovation, we aspire to catalyze future advancements in Explainable AI, ultimately paving the way towards more accountable and trustworthy AI systems.

Facts and Hopes in Using Omics to Advance Combined Immunotherapy Strategies.

The field of oncology has been transformed by immune checkpoint inhibitors (ICI) and other immune-based agents; however, many patients do not receive a durable benefit. While biomarker assessments from pivotal ICI trials have uncovered certain mechanisms of resistance, results thus far have only scraped the surface. Mechanisms of resistance are as complex as the tumor microenvironment (TME) itself, and the development of effective therapeutic strategies will only be possible by building accurate models of the tumor-immune interface. With advancement of multi-omic technologies, high-resolution characterization of the TME is now possible. In addition to sequencing of bulk tumor, single-cell transcriptomic, proteomic, and epigenomic data as well as T-cell receptor profiling can now be simultaneously measured and compared between responders and nonresponders to ICI. Spatial sequencing and imaging platforms have further expanded the dimensionality of existing technologies. Rapid advancements in computation and data sharing strategies enable development of biologically interpretable machine learning models to integrate data from high-resolution, multi-omic platforms. These models catalyze the identification of resistance mechanisms and predictors of benefit in ICI-treated patients, providing scientific foundation for novel clinical trials. Moving forward, we propose a framework by which in silico screening, functional validation, and clinical trial biomarker assessment can be used for the advancement of combined immunotherapy strategies.

The interpretable machine learning model associated with metal mixtures to identify hypertension via EMR mining method.

There are limited data available regarding the connection between hypertension and heavy metal exposure. The authors intend to establish an interpretable machine learning (ML) model with high efficiency and robustness that identifies hypertension based on heavy metal exposure. Our datasets were obtained from the US National Health and Nutrition Examination Survey (NHANES, 2013-2020.3). The authors developed 5 ML models for hypertension identification by heavy metal exposure, and tested them by 10 discrimination characteristics. Further, the authors chose the optimally performing model after parameter adjustment by Genetic Algorithm (GA) for identification. Finally, in order to visualize the model's ability to make decisions, the authors used SHapley Additive exPlanation (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME) algorithm to illustrate the features. The study included 19368 participants in total. A best-performing eXtreme Gradient Boosting (XGB) with GA for hypertension identification by 16 heavy metals was selected (AUC: 0.774; 95% CI: 0.772-0.776; accuracy: 87.7%). According to SHAP values, Barium (0.02), Cadmium (0.017), Lead (0.017), Antimony (0.008), Tin (0.007), Manganese (0.006), Thallium (0.004), Tungsten (0.004) in urine, and Lead (0.048), Mercury (0.035), Selenium (0.05), Manganese (0.007) in blood positively influenced the model, while Cadmium (-0.001) in urine negatively influenced the model. Study participants' hypertension associated with heavy metal exposure was identified by an efficient, robust, and interpretable GA-XGB model with SHAP and LIME. Barium, Cadmium, Lead, Antimony, Tin, Manganese, Thallium, Tungsten in urine, and Lead, Mercury, Selenium, Manganese in blood are positively correlated with hypertension, while Cadmium in blood is negatively correlated with hypertension.

PM2.5 pollution modulates the response of ozone formation to VOC emitted from various sources: Insights from machine learning

Numerous studies have linked ozone (O3) production to its precursors and fine particulate matter (PM2.5), while the complex interaction effects of PM2.5 and volatile organic compounds (VOCs) on O3 remain poorly understood. A systematic approach based on an interpretable machine learning (ML) model was utilized to evaluate the primary driving factors that impact O3 and to elucidate how changes in PM2.5, VOCs from different sources, NOx, and meteorological conditions either promote or inhibit O3 formation through their individual and synergistic effects in a tropical coastal city, Haikou, from 2019 to 2020. The results suggest that under low PM2.5 levels, alongside the linear O3-PM2.5 relationship observed, O3 formation is suppressed by PM2.5 with higher proportions of traffic-derived aerosol. Vehicle VOC emissions contributed maximally to O3 formation at midday, despite the lowest concentration. VOCs from fossil fuel combustion and industry emissions, which have opposing effects on O3, act as inhibitors and promoters by inducing diverse photochemical regimes. As PM2.5 pollution escalates, the impact of these VOCs reverses, becoming more pronounced in shaping O3 variation. Sensitivity analysis reveals that the O3 formation regime is VOC-limited, and effective regional O3 mitigation requires prioritizing substantial VOC reductions to offset enhanced VOC sensitivity induced by the co-reduction in PM2.5, with a focus on industrial and vehicular emissions, and subsequently, fossil fuel combustion once PM2.5 is effectively controlled. This study underscores the potential of the SHAP-based ML approach to decode the intricate O3-NOx-VOCs-PM2.5 interplay, considering both meteorological and atmospheric compositional variations.

Interpretable machine learning models for predicting in-hospital and 30 days adverse events in acute coronary syndrome patients in Kuwait

The relationships between acute coronary syndromes (ACS) adverse events and the associated risk factors are typically complicated and nonlinear, which poses significant challenges to clinicians' attempts at risk stratification. Here, we aim to explore the implementation of modern risk stratification tools to untangle how these complex factors shape the risk of adverse events in patients with ACS. We used an interpretable multi-algorithm machine learning (ML) approach and clinical features to fit predictive models to 1,976 patients with ACS in Kuwait. We demonstrated that random forest (RF) and extreme gradient boosting (XGB) algorithms, remarkably outperform traditional logistic regression model (AUCs = 0.84 & 0.79 for RF and XGB, respectively). Our in-hospital adverse events model identified left ventricular ejection fraction as the most important predictor with the highest interaction strength with other factors. However, using the 30-days adverse events model, we found that performing an urgent coronary artery bypass graft was the most important predictor, with creatinine levels having the strongest overall interaction with other related factors. Our ML models not only untangled the non-linear relationships that shape the clinical epidemiology of ACS adverse events but also elucidated their risk in individual patients based on their unique features.

Enhancing winter road maintenance with explainable AI: SHAP analysis for interpreting machine learning models in road friction estimation

Effective winter road maintenance relies on precise road friction estimation. Machine learning (ML) models have shown significant promise in this; however, their inherent complexity makes understanding their inner workings challenging. This paper addresses this issue by conducting a comparative analysis of road friction estimation models using four ML methods, including regression tree, random forest, eXtreme Gradient Boosting (XGBoost), and support vector regression (SVR). We then employ the SHapley Additive exPlanations (SHAP) explainable artificial intelligence (AI) to enhance model interpretability. Our analysis on an Alberta dataset reveals that the XGBoost model performs best with an accuracy of 91.39%. The SHAP analysis illustrates the logical relationships between predictor features and friction within all three tree-based models, but it also uncovers inconsistencies within the SVR model, potentially attributed to insufficient feature interactions. Thus, this paper not only showcase the role of explainable AI in improving the ML interpretability of models for road friction estimation, but also provides practical insights that could improve winter road maintenance decisions.

Interpretable Medical Imagery Diagnosis with Self-Attentive Transformers: A Review of Explainable AI for Health Care

Recent advancements in artificial intelligence (AI) have facilitated its widespread adoption in primary medical services, addressing the demand–supply imbalance in healthcare. Vision Transformers (ViT) have emerged as state-of-the-art computer vision models, benefiting from self-attention modules. However, compared to traditional machine learning approaches, deep learning models are complex and are often treated as a “black box” that can cause uncertainty regarding how they operate. Explainable artificial intelligence (XAI) refers to methods that explain and interpret machine learning models’ inner workings and how they come to decisions, which is especially important in the medical domain to guide healthcare decision-making processes. This review summarizes recent ViT advancements and interpretative approaches to understanding the decision-making process of ViT, enabling transparency in medical diagnosis applications.

An interpretable machine learning model for stroke recurrence in patients with symptomatic intracranial atherosclerotic arterial stenosis.

Symptomatic intracranial atherosclerotic stenosis (SICAS) is the most common etiology of ischemic stroke and one of the main causes of high stroke recurrence. The recurrence of stroke is closely related to the prognosis of ischemic stroke. This study aims to develop a machine learning model based on high-resolution vessel wall imaging (HR-VWI) to predict the risk of stroke recurrence in SICAS. This study retrospectively collected data from 180 SICAS stroke patients treated at the hospital between 2020.01 and 2022.01. Relevant imaging and clinical data were collected, and follow-up was conducted. The dataset was divided into a training set and a validation set in a ratio of 7:3. We employed the least absolute shrinkage and selection operator (LASSO) regression to perform a selection on the baseline data, laboratory tests, and neuroimaging data generated by HR-VWI scans collected from the training set. Finally, five machine learning techniques, including logistic regression model (LR), support vector machine (SVM), Gaussian naive Bayes (GNB), Complement naive Bayes (CNB), and k-nearest neighbors algorithm (kNN), were employed to develop a predictive model for stroke recurrence. Shapley Additive Explanation (SHAP) was used to provide visualization and interpretation for each patient. The model's effectiveness was evaluated using average accuracy, sensitivity, specificity, precision, f1 score, PR curve, calibration curve, and decision curve analysis. LASSO analysis revealed that "history of hypertension," "homocysteine level," "NWI value," "stenosis rate," "intracranial hemorrhage," "positive remodeling," and "enhancement grade" were independent risk factors for stroke recurrence in SICAS patients. In 10-fold cross-validation, the area under the curve (AUC) ranged from 0.813 to 0.912 in ROC curve analysis. The area under the precision-recall curve (AUPRC) ranged from 0.655 to 0.833, with the Gaussian Naive Bayes (GNB) model exhibiting the best ability to predict stroke recurrence in SICAS. SHAP analysis provided interpretability for the machine learning model and revealed essential factors related to the risk of stroke recurrence in SICAS. A precise machine learning-based prediction model for stroke recurrence in SICAS has been established to assist clinical practitioners in making clinical decisions and implementing personalized treatment measures.

Synergistic observation of FY-4A&4B to estimate CO concentration in China: combining interpretable machine learning to reveal the influencing mechanisms of CO variations

Accurately estimating the concentration of carbon monoxide (CO) with high spatiotemporal resolution is crucial for assessing its meteorological-environmental-health impacts. Although machine learning models have high predictive ability in environmental research, there are relatively few explanations for model outputs. Utilizing the top-of-atmosphere radiation data of China’s new generation geostationary satellites (FY-4A and FY-4B) and interpretable machine learning models, the 24-hour near-surface CO concentrations in China was conducted (resolution: 1 hour, 0.04°). The model improved by 6.6% when using the all-sky dataset (cloud-contained model, R2 = 0.759) compared to the clear-sky dataset (cloud-removed model). The interpretability analysis of the CO estimation model used two methods, namely ante-hoc (model feature importance) and post-hoc (SHapley Additive exPlanations). The importance of daytime meteorological factors increased by 51% compared to nighttime. Combining partial dependency plots, the impact of key meteorological factors on CO was elucidated to gain a deeper understanding of the spatiotemporal variations of CO.

InterpretME: A tool for interpretations of machine learning models over knowledge graphs

In recent years, knowledge graphs (KGs) have been considered pyramids of interconnected data enriched with semantics for complex decision-making. The potential of KGs and the demand for interpretability of machine learning (ML) models in diverse domains (e.g., healthcare) have gained more attention. The lack of model transparency negatively impacts the understanding and, in consequence, interpretability of the predictions made by a model. Data-driven models should be empowered with the knowledge required to trace down their decisions and the transformations made to the input data to increase model transparency. In this paper, we propose InterpretME, a tool that using KGs, provides fine-grained representations of trained ML models. An ML model description includes data – (e.g., features’ definition and SHACL validation) and model-based characteristics (e.g., relevant features and interpretations of prediction probabilities and model decisions). InterpretME allows for defining a model’s features over data collected in various formats, e.g., RDF KGs, CSV, and JSON. InterpretME relies on the SHACL schema to validate integrity constraints over the input data. InterpretME traces the steps of data collection, curation, integration, and prediction; it documents the collected metadata in the InterpretME KG. InterpretME is published in GitHub11 https://github.com/SDM-TIB/InterpretME and Zenodo22 https://doi.org/10.5281/zenodo.8112628. The InterpretME framework includes a pipeline for enhancing the interpretability of ML models, the InterpretME KG, and an ontology to describe the main characteristics of trained ML models; a PyPI library of InterpretME is also provided33 https://pypi.org/project/InterpretME/. Additionally, a live code44 https://github.com/SDM-TIB/InterpretME_Demo, and a video55 https://www.youtube.com/watch?v=Bu4lROnY4xg demonstrating InterpretME in several use cases are also available.

Interpretable machine learning model for early prediction of 28-day mortality in ICU patients with sepsis-induced coagulopathy: development and validation

ObjectiveSepsis-induced coagulopathy (SIC) is extremely common in individuals with sepsis, significantly associated with poor outcomes. This study attempted to develop an interpretable and generalizable machine learning (ML) model for early predicting the risk of 28-day death in patients with SIC.MethodsIn this retrospective cohort study, we extracted SIC patients from the Medical Information Mart for Intensive Care III (MIMIC-III), MIMIC-IV, and eICU-CRD database according to Toshiaki Iba's scale. And the overlapping in the MIMIC-IV was excluded for this study. Afterward, only the MIMIC-III cohort was randomly divided into the training set, and the internal validation set according to the ratio of 7:3, while the MIMIC-IV and eICU-CRD databases were considered the external validation sets. The predictive factors for 28-day mortality of SIC patients were determined using recursive feature elimination combined with tenfold cross-validation (RFECV). Then, we constructed models using ML algorithms. Multiple metrics were used for evaluation of performance of the models, including the area under the receiver operating characteristic curve (AUROC), area under the precision recall curve (AUPRC), accuracy, sensitivity, specificity, negative predictive value, positive predictive value, recall, and F1 score. Finally, Shapley Additive Explanations (SHAP), Local Interpretable Model-Agnostic Explanations (LIME) were employed to provide a reasonable interpretation for the prediction results.ResultsA total of 3280, 2798, and 1668 SIC patients were screened from MIMIC-III, MIMIC-IV, and eICU-CRD databases, respectively. Seventeen features were selected to construct ML prediction models. XGBoost had the best performance in predicting the 28-day mortality of SIC patients, with AUC of 0.828, 0.913 and 0.923, the AUPRC of 0.807, 0.796 and 0.921, the accuracy of 0.785, 0.885 and 0.891, the F1 scores were 0.63, 0.69 and 0.70 in MIMIC-III (internal validation set), MIMIC-IV, and eICU-CRD databases. The importance ranking and SHAP analyses showed that initial SOFA score, red blood cell distribution width (RDW), and age were the top three critical features in the XGBoost model.ConclusionsWe developed an optimal and explainable ML model to predict the risk of 28-day death of SIC patients 28-day death risk. Compared with conventional scoring systems, the XGBoost model performed better. The model established will have the potential to improve the level of clinical practice for SIC patients.Graphical

Wearable sensor devices can automatically identify the ON-OFF status of patients with Parkinson's disease through an interpretable machine learning model.

Accurately and objectively quantifying the clinical features of Parkinson's disease (PD) is crucial for assisting in diagnosis and guiding the formulation of treatment plans. Therefore, based on the data on multi-site motor features, this study aimed to develop an interpretable machine learning (ML) model for classifying the "OFF" and "ON" status of patients with PD, as well as to explore the motor features that are most associated with changes in clinical symptoms. We employed a support vector machine with a recursive feature elimination (SVM-RFE) algorithm to select promising motion features. Subsequently, 12 ML models were constructed based on these features, and we identified the model with the best classification performance. Then, we used the SHapley Additive exPlanations (SHAP) and the Local Interpretable Model agnostic Explanations (LIME) methods to explain the model and rank the importance of those motor features. A total of 96 patients were finally included in this study. The naive Bayes (NB) model had the highest classification performance (AUC = 0.956; sensitivity = 0.8947, 95% CI 0.6686-0.9870; accuracy = 0.8421, 95% CI 0.6875-0.9398). Based on the NB model, we analyzed the importance of eight motor features toward the classification results using the SHAP algorithm. The Gait: range of motion (RoM) Shank left (L) (degrees) [Mean] might be the most important motor feature for all classification horizons. The symptoms of PD could be objectively quantified. By utilizing suitable motor features to construct ML models, it became possible to intelligently identify whether patients with PD were in the "ON" or "OFF" status. The variations in these motor features were significantly correlated with improvement rates in patients' quality of life. In the future, they might act as objective digital biomarkers to elucidate the changes in symptoms observed in patients with PD and might be used to assist in the diagnosis and treatment of patients with PD.

A Visual Analytics Conceptual Framework for Explorable and Steerable Partial Dependence Analysis.

Machine learning techniques are a driving force for research in various fields, from credit card fraud detection to stock analysis. Recently, a growing interest in increasing human involvement has emerged, with the primary goal of improving the interpretability of machine learning models. Among different techniques, Partial Dependence Plots (PDP) represent one of the main model-agnostic approaches for interpreting how the features influence the prediction of a machine learning model. However, its limitations (i.e., visual interpretation, aggregation of heterogeneous effects, inaccuracy, and computability) could complicate or misdirect the analysis. Moreover, the resulting combinatorial space can be challenging to explore both computationally and cognitively when analyzing the effects of more features at the same time. This paper proposes a conceptual framework that enables effective analysis workflows, mitigating state-of-the-art limitations. The proposed framework allows for exploring and refining computed partial dependences, observing incrementally accurate results, and steering the computation of new partial dependences on user-selected subspaces of the combinatorial and intractable space. With this approach, the user can save both computational and cognitive costs, in contrast with the standard monolithic approach that computes all the possible combinations of features on all their domains in batch. The framework is the result of a careful design process involving experts' knowledge during its validation and informed the development of a prototype, W4SP (available at https://aware-diag-sapienza.github.io/W4SP/), that demonstrates its applicability traversing its different paths. A case study shows the advantages of the proposed approach.

XAI-PDF: A Robust Framework for Malicious PDF Detection Leveraging SHAP-Based Feature Engineering

With the increasing number of malicious PDF files used for cyberattacks, it is essential to develop efficient and accurate classifiers to detect and prevent these threats. Machine Learning (ML) models have successfully detected malicious PDF files. This paper presents XAI-PDF, an efficient system for malicious PDF detection designed to enhance accuracy and minimize decision-making time on a modern dataset, the Evasive-PDFMal2022 dataset. The proposed method optimizes malicious PDF classifier performance by employing feature engineering guided by Shapley Additive Explanations (SHAP). Particularly, the model development approach comprises four phases: data preparation, model building, explainability of the models, and derived features. Utilizing the interpretability of SHAP values, crucial features are identified, and new ones are generated, resulting in an improved classification model that showcases the effectiveness of interpretable AI techniques in enhancing model performance. Various interpretable ML models were implemented, with the Lightweight Gradient Boosting Machine (LGBM) outperforming other classifiers. The Explainable Artificial Intelligence (XAI) global surrogate model generated explanations for LGBM predictions. Experimental comparisons of XAI-PDF with baseline methods revealed its superiority in achieving higher accuracy, precision, and F1-scores with minimal False Positive (FP) and False Negative (FN) rates (99.9%, 100%, 99.89%,0.000, and 0.002, respectively). Additionally, XAI-PDF requires only 1.36 milliseconds per record for predictions, demonstrating increased resilience in detecting evasive malicious PDF files compared to state-of-the-art methods