Model-agnostic Methods Research Articles

Regional-scale prediction of pluvial and flash flood susceptible areas using tree-based classifiers

• Literature review on explanatory features used in flash flood modeling. • Comparison of three tree-based classifiers for a large investigation area. • Use of 11 spatially distributed and 6 catchment-related explanatory features. • Model interpretation using model-specific and model-agnostic methods. Flood events triggered by heavy rain, such as pluvial and flash floods, are a common threat worldwide. However, it is usually not known which areas and cities are particularly vulnerable to flooding caused by heavy rain. To enable regional-scale susceptibility assessment, we developed a novel methodology based on tree-based classifiers that considers both spatially distributed and catchment-related influencing factors. The performance of the developed methodology was evaluated using the region of Bavaria (Germany). For the case study area (70,500 km 2 ), we trained a Random Forest (RF), a Gradient Boosting Decision Tree (GBDT), and a CatBoost model (CB) using 1,864 flood and non-flood locations and 11 spatially distributed and six catchment-related influencing factors. Regarding performance metrics, all three models performed equally well (CB: AUC = 0.819, RF: AUC = 0.816, GBDT: AUC = 0.813), with the CatBoost model performing best. When modeling large areas, it proved critical to account for low sample density by ensuring (i) a homogeneous spatial coverage of the study area and (ii) the representation of the major landscapes in the training and test set. In addition, we propose an overall susceptibility score for cities based on the susceptibility map generated, which can be used to prioritize cities for detailed investigations. Those responsible for spatial planning and flood risk management can apply the proposed methodology to obtain a pluvial and flash flood susceptibility assessment for large territories.

A Case Study on the Application of Model-agnostic Methods for the Post hoc Interpretation of A Machine Learning Model :

A Case Study on the Application of Model-agnostic Methods for the Post hoc Interpretation of A Machine Learning Model :

Decision Support System Improving the Interpretability of Generated Tree-Based Models

Abstract A decision tree represents one of the most used data analysis methods for classification tasks. The generated decision models can be visualized as a graph, but this visualization is quite complicated for a domain expert to understand in large or heterogeneous data. Our previous experience with medical data analytics related to the classification of patients with Metabolic Syndrome, Mild Cognitive Impairment, heart disease, or Frailty motivated us to evaluate the potential of new visualizations for this decision model in the medical domain. We managed a user study to design and implement a decision support system containing selected methods to improve the interpretability of the generated tree-based decision model. We hypothesized that this approach would result in more effective communication between data analysts and medical experts, reduce necessary time and energy and bring more comprehensive results. For this purpose, we selected two model-agnostic methods, LIME and SHAP, and one new interactive visualization called Sunburst. We used two data samples for design and evaluation: the publicly available heart disease dataset and the Metabolic Syndrome dataset the participating medical expert provided. We will use the collected feedback and experience for further improvements, like more evaluation metrics related to the usability of the decision models.

Believe the HiPe: Hierarchical perturbation for fast, robust, and model-agnostic saliency mapping

Understanding the predictions made by Artificial Intelligence (AI) systems is becoming more and more important as deep learning models are used for increasingly complex and high-stakes tasks. Saliency mapping – a popular visual attribution method – is one important tool for this, but existing formulations are limited by either computational cost or architectural constraints. We therefore propose Hierarchical Perturbation, a very fast and completely model-agnostic method for interpreting model predictions with robust saliency maps. Using standard benchmarks and datasets, we show that our saliency maps are of competitive or superior quality to those generated by existing model-agnostic methods – and are over 20× faster to compute.

Multicriteria interpretability driven deep learning

AbstractDeep Learning methods are well-known for their abilities, but their interpretability keeps them out of high-stakes situations. This difficulty is addressed by recent model-agnostic methods that provide explanations after the training process. As a result, the current guidelines’ requirement for “interpretability from the start” is not met. As a result, such methods are only useful as a sanity check after the model has been trained. In an abstract scenario, “interpretability from the start” implies imposing a set of soft constraints on the model’s behavior by infusing knowledge and eliminating any biases. By inserting knowledge into the objective function, we present a Multicriteria technique that allows us to control the feature effects on the model’s output. To accommodate for more complex effects and local lack of information, we enhance the method by integrating particular knowledge functions. As a result, a Deep Learning training process that is both interpretable and compliant with modern legislation has been developed. Our technique develops performant yet robust models capable of overcoming biases resulting from data scarcity, according to a practical empirical example based on credit risk.

Counterfactuals and causability in explainable artificial intelligence: Theory, algorithms, and applications

Deep learning models have achieved high performance across different domains, such as medical decision-making, autonomous vehicles, decision support systems, among many others. However, despite this success, the inner mechanisms of these models are opaque because their internal representations are too complex for a human to understand. This opacity makes it hard to understand the how or the why of the predictions of deep learning models.There has been a growing interest in model-agnostic methods that make deep learning models more transparent and explainable to humans. Some researchers recently argued that for a machine to achieve human-level explainability, this machine needs to provide human causally understandable explanations, also known as causability. A specific class of algorithms that have the potential to provide causability are counterfactuals.This paper presents an in-depth systematic review of the diverse existing literature on counterfactuals and causability for explainable artificial intelligence (AI). We performed a Latent Dirichlet topic modelling analysis (LDA) under a Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework to find the most relevant literature articles. This analysis yielded a novel taxonomy that considers the grounding theories of the surveyed algorithms, together with their underlying properties and applications to real-world data.Our research suggests that current model-agnostic counterfactual algorithms for explainable AI are not grounded on a causal theoretical formalism and, consequently, cannot promote causability to a human decision-maker. Furthermore, our findings suggest that the explanations derived from popular algorithms in the literature provide spurious correlations rather than cause/effects relationships, leading to sub-optimal, erroneous, or even biased explanations. Thus, this paper also advances the literature with new directions and challenges on promoting causability in model-agnostic approaches for explainable AI.

A Quantitative Evaluation of Global, Rule-Based Explanations of Post-Hoc, Model Agnostic Methods

Understanding the inferences of data-driven, machine-learned models can be seen as a process that discloses the relationships between their input and output. These relationships consist and can be represented as a set of inference rules. However, the models usually do not explicit these rules to their end-users who, subsequently, perceive them as black-boxes and might not trust their predictions. Therefore, scholars have proposed several methods for extracting rules from data-driven machine-learned models to explain their logic. However, limited work exists on the evaluation and comparison of these methods. This study proposes a novel comparative approach to evaluate and compare the rulesets produced by five model-agnostic, post-hoc rule extractors by employing eight quantitative metrics. Eventually, the Friedman test was employed to check whether a method consistently performed better than the others, in terms of the selected metrics, and could be considered superior. Findings demonstrate that these metrics do not provide sufficient evidence to identify superior methods over the others. However, when used together, these metrics form a tool, applicable to every rule-extraction method and machine-learned models, that is, suitable to highlight the strengths and weaknesses of the rule-extractors in various applications in an objective and straightforward manner, without any human interventions. Thus, they are capable of successfully modelling distinctively aspects of explainability, providing to researchers and practitioners vital insights on what a model has learned during its training process and how it makes its predictions.

Framework for collaborative intelligence in forecasting day-ahead electricity price

Electricity price forecasting in wholesale markets is an essential asset for deciding bidding strategies and operational schedules. The decision making process is limited if no understanding is given on how and why such electricity price points have been forecast. The present article proposes a novel framework that promotes human–machine collaboration in forecasting day-ahead electricity price in wholesale markets. The framework is based on a new model architecture that uses a plethora of statistical and machine learning models, a wide range of exogenous features, a combination of several time series decomposition methods and a collection of time series characteristics based on signal processing and time series analysis methods. The model architecture is supported by open-source automated machine learning platforms that provide a baseline reference used for comparison purposes. The objective of the framework is not only to provide forecasts, but to promote a human-in-the-loop approach by providing a data story based on a collection of model-agnostic methods aimed at interpreting the mechanisms and behavior of the new model architecture and its predictions. The framework has been applied to the Spanish wholesale market. The forecasting results show good accuracy on mean absolute error (1.859, 95% HDI [0.575, 3.924] EUR(MWh)−1) and mean absolute scaled error (0.378, 95% HDI [0.091, 0.934]). Moreover, the framework demonstrates its human-centric capabilities by providing graphical and numeric explanations that augments understanding on the model and its electricity price point forecasts.

Novel Application of Machine Learning Algorithms and Model-Agnostic Methods to Identify Factors Influencing Childhood Blood Lead Levels.

Blood lead (Pb) poisoning remains a global concern, particularly for children in their early developmental years. Broken Hill is Australia's oldest operating silver-zinc-lead mine. In this study, we utilized recent advances in machine learning to assess multiple algorithms and identify the most optimal model for predicting childhood blood Pb levels (BLL) using Broken Hill children's (<5 years of age) data (n = 23,749) from 1991 to 2015, combined with demographic, socio-economic, and environmental influencing factors. We applied model-agnostic methods to interpret the most optimal model, investigating different environmental and human factors influencing childhood BLL. Algorithm assessment showed that stacked ensemble, a method for automatically and optimally combining multiple prediction algorithms, enhanced predictive performance by 1.1% with respect to mean absolute error (p < 0.01) and 2.6% for root-mean-squared error (p < 0.01) compared to the best performing constituent algorithm (random forest). By interpreting the model, the following information was acquired: children had higher BLL if they resided within 1.0 km to the central mine area or 1.37 km to the railroad; year of testing had the greatest interactive strength with all other factors; BLL increased faster in Aboriginal than in non-Aboriginal children at 9-10 and 12-18 months of age. This "stacked ensemble + model-agnostic interpretation" framework achieved both prediction accuracy and model interpretability, identifying previously unconnected variables associated with elevated childhood BLL, offering a marked advantage over previous works. Thus, this approach has a clear value and potential for application to other environmental health issues.

Regularizing Attention Networks for Anomaly Detection in Visual Question Answering

For stability and reliability of real-world applications, the robustness of DNNs in unimodal tasks has been evaluated. However, few studies consider abnormal situations that a visual question answering (VQA) model might encounter at test time after deployment in the real-world. In this study, we evaluate the robustness of state-of-the-art VQA models to five different anomalies, including worst-case scenarios, the most frequent scenarios, and the current limitation of VQA models. Different from the results in unimodal tasks, the maximum confidence of answers in VQA models cannot detect anomalous inputs, and post-training of the outputs, such as outlier exposure, is ineffective for VQA models. Thus, we propose an attention-based method, which uses confidence of reasoning between input images and questions and shows much more promising results than the previous methods in unimodal tasks. In addition, we show that a maximum entropy regularization of attention networks can significantly improve the attention-based anomaly detection of the VQA models. Thanks to the simplicity, attention-based anomaly detection and the regularization are model-agnostic methods, which can be used for various cross-modal attentions in the state-of-the-art VQA models. The results imply that cross-modal attention in VQA is important to improve not only VQA accuracy, but also the robustness to various anomalies.

Interpretable heartbeat classification using local model-agnostic explanations on ECGs

Treatment and prevention of cardiovascular diseases often rely on Electrocardiogram (ECG) interpretation. Dependent on the physician's variability, ECG interpretation is subjective and prone to errors. Machine learning models are often developed and used to support doctors; however, their lack of interpretability stands as one of the main drawbacks of their widespread operation. This paper focuses on an Explainable Artificial Intelligence (XAI) solution to make heartbeat classification more explainable using several state-of-the-art model-agnostic methods. We introduce a high-level conceptual framework for explainable time series and propose an original method that adds temporal dependency between time samples using the time series' derivative. The results were validated in the MIT-BIH arrhythmia dataset: we performed a performance's analysis to evaluate whether the explanations fit the model's behaviour; and employed the 1-D Jaccard's index to compare the subsequences extracted from an interpretable model and the XAI methods used. Our results show that the use of the raw signal and its derivative includes temporal dependency between samples to promote classification explanation. A small but informative user study concludes this study to evaluate the potential of the visual explanations produced by our original method for being adopted in real-world clinical settings, either as diagnostic aids or training resource.

Improving robustness of deep neural networks via large-difference transformation

Recent research shows that previous model-agnostic methods that transform the input images before feeding them into the classifiers fail to defend against the adversarial examples. We assume that the small-difference transformations commonly used are the blame and therefore propose a new model-agnostic defense using a large-difference transformation. Specifically, we try to apply the novel primitive-based transformation that re-builds the input images by primitives of colorful triangles. In terms of the distortions required to completely break the defenses, our experiments on the ImageNet subset show that significantly large distortions (0.12) are needed to break the defense compared to other state-of-the-art model-agnostic defenses (0.05–0.06) under the strong attack method Backward Pass Differentiable Approximation (BPDA). This finding indicates that large difference transformation can improve the adversarial robustness, suggesting a promising new direction towards solving the challenge of adversarial robustness.

Comparison of prices depending on factors in the secondary housing market

The article considers the issue of pricing for the secondary real estate market regarding local causes (physical properties of housing). The aim of the study was to verify the following hypotheses: the influence of the pricing factors of residential real estate on its value is determined by its price segment and the influence of infrastructure on the value of apartments in different cities is the same. In the hypothesis test, data were used on the secondary housing market of the cities of Novosibirsk and Krasnoyarsk, taken from the site of «CIAN» apartment sale announcements and from various open data sources. During the study, non-parametric methods of machine learning, model-agnostic methods for the interpretation of predictive models, hierarchical clustering are involved. As a result of the work, the first hypothesis was confirmed and the second hypothesis was refuted, the high accuracy of forecasting the cost of an apartment was achieved, and the peculiarities of price formation for secondary housing objects were revealed and described.

Predicting dengue importation into Europe, using machine learning and model-agnostic methods

The geographical spread of dengue is a global public health concern. This is largely mediated by the importation of dengue from endemic to non-endemic areas via the increasing connectivity of the global air transport network. The dynamic nature and intrinsic heterogeneity of the air transport network make it challenging to predict dengue importation. Here, we explore the capabilities of state-of-the-art machine learning algorithms to predict dengue importation. We trained four machine learning classifiers algorithms, using a 6-year historical dengue importation data for 21 countries in Europe and connectivity indices mediating importation and air transport network centrality measures. Predictive performance for the classifiers was evaluated using the area under the receiving operating characteristic curve, sensitivity, and specificity measures. Finally, we applied practical model-agnostic methods, to provide an in-depth explanation of our optimal model’s predictions on a global and local scale. Our best performing model achieved high predictive accuracy, with an area under the receiver operating characteristic score of 0.94 and a maximized sensitivity score of 0.88. The predictor variables identified as most important were the source country’s dengue incidence rate, population size, and volume of air passengers. Network centrality measures, describing the positioning of European countries within the air travel network, were also influential to the predictions. We demonstrated the high predictive performance of a machine learning model in predicting dengue importation and the utility of the model-agnostic methods to offer a comprehensive understanding of the reasons behind the predictions. Similar approaches can be utilized in the development of an operational early warning surveillance system for dengue importation.

Controlled feature selection and compressive big data analytics: Applications to biomedical and health studies.

The theoretical foundations of Big Data Science are not fully developed, yet. This study proposes a new scalable framework for Big Data representation, high-throughput analytics (variable selection and noise reduction), and model-free inference. Specifically, we explore the core principles of distribution-free and model-agnostic methods for scientific inference based on Big Data sets. Compressive Big Data analytics (CBDA) iteratively generates random (sub)samples from a big and complex dataset. This subsampling with replacement is conducted on the feature and case levels and results in samples that are not necessarily consistent or congruent across iterations. The approach relies on an ensemble predictor where established model-based or model-free inference techniques are iteratively applied to preprocessed and harmonized samples. Repeating the subsampling and prediction steps many times, yields derived likelihoods, probabilities, or parameter estimates, which can be used to assess the algorithm reliability and accuracy of findings via bootstrapping methods, or to extract important features via controlled variable selection. CBDA provides a scalable algorithm for addressing some of the challenges associated with handling complex, incongruent, incomplete and multi-source data and analytics challenges. Albeit not fully developed yet, a CBDA mathematical framework will enable the study of the ergodic properties and the asymptotics of the specific statistical inference approaches via CBDA. We implemented the high-throughput CBDA method using pure R as well as via the graphical pipeline environment. To validate the technique, we used several simulated datasets as well as a real neuroimaging-genetics of Alzheimer’s disease case-study. The CBDA approach may be customized to provide generic representation of complex multimodal datasets and to provide stable scientific inference for large, incomplete, and multisource datasets.