“I do not know! but why?” — Local model-agnostic example-based explanations of reject

Abstract

Machine learning based decision making systems in safety critical areas place high demands on the accuracy and generalization ability of the underlying model. A common strategy to deal with uncertainties and possible mistakes is offered by learning with reject option, i.e. a model can refrain from prediction in ambiguous cases and leave the decision to a human expert. Yet, as for the models themselves, human decision-making is hampered by the fact that reject options are often implemented as black-box rules: Experts cannot readily understand the reasons for rejection.In this work, we propose a model-agnostic framework that enriches classification with reject option by explanation mechanisms. More specifically, we combine conformal prediction as a popular mathematically based technology of certainty estimation with local surrogates derived for the region of interest. This allows us to provide local explanations in terms of example-based explanation methods, including counterfactual, semi-factual, and factual methods. We demonstrate the performance of this technology through a series of benchmarks using 6 different data sets; the associated code is open source.11This article represents a substantially extended version of the conference paper Artelt et al. (2022). In addition to a more detailed explanation and derivation of the algorithms, we explore semi-factual and factual explanation methods and provide evaluations on realistic benchmark data sets.