Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics

Jianlong Zhou,Andreas Holzinger,Amir H Gandomi,Fang Chen

doi:10.3390/electronics10050593

Jianlong Zhou, Andreas Holzinger + Show 2 more

Open Access

https://doi.org/10.3390/electronics10050593

Copy DOI

Abstract

The most successful Machine Learning (ML) systems remain complex black boxes to end-users, and even experts are often unable to understand the rationale behind their decisions. The lack of transparency of such systems can have severe consequences or poor uses of limited valuable resources in medical diagnosis, financial decision-making, and in other high-stake domains. Therefore, the issue of ML explanation has experienced a surge in interest from the research community to application domains. While numerous explanation methods have been explored, there is a need for evaluations to quantify the quality of explanation methods to determine whether and to what extent the offered explainability achieves the defined objective, and compare available explanation methods and suggest the best explanation from the comparison for a specific task. This survey paper presents a comprehensive overview of methods proposed in the current literature for the evaluation of ML explanations. We identify properties of explainability from the review of definitions of explainability. The identified properties of explainability are used as objectives that evaluation metrics should achieve. The survey found that the quantitative metrics for both model-based and example-based explanations are primarily used to evaluate the parsimony/simplicity of interpretability, while the quantitative metrics for attribution-based explanations are primarily used to evaluate the soundness of fidelity of explainability. The survey also demonstrated that subjective measures, such as trust and confidence, have been embraced as the focal point for the human-centered evaluation of explainable systems. The paper concludes that the evaluation of ML explanations is a multidisciplinary research topic. It is also not possible to define an implementation of evaluation metrics, which can be applied to all explanation methods.

Highlights

Machine Learning (ML) systems are increasingly used in various fields and becoming increasingly capable of solving different tasks range from the everyday life assistant to decision-making in high-stake domains
It is interesting to find that the quantitative metrics for both model-based and example-based explanations are primarily used to evaluate the parsimony/simplicity of interpretability, while the quantitative metrics for attribution-based explanations are primarily used to evaluate the soundness of fidelity of explainability
We connected the properties of explainability with categories of ML explanation methods

Summary

Introduction

Machine Learning (ML) systems are increasingly used in various fields and becoming increasingly capable of solving different tasks range from the everyday life assistant (smart health) to decision-making in high-stake domains (clinical decision support). Because of the black box nature of ML models [6,7], the deployment of ML algorithms, especially in high stake domains, such as medical diagnosis, criminal justice, financial decision-making, and other regulated safety critical domains, requires verification and testing for plausibility by domain experts for safety but for legal reasons [8]. Users want to understand reasons behind specific decisions based on ML models. Such requirements result in high societal and ethical demands to provide explanations for such ML systems. ML explanations are becoming indispensable to interpret black box results and to allow users to gain insights into the system’s decision-making process, which is a key component in fostering trust and confidence in ML systems [9,10,11]

Objectives

Results

Discussion

Conclusion