Explainability in process outcome prediction: Guidelines to obtain interpretable and faithful models

Abstract

Process outcome prediction pertains to the classification of ongoing cases of (business) processes into a given set of categorical outcomes. This field of research has seen a strong uptake in recent years due to advances in machine and deep learning. Although a recent shift has been made in the field of process outcome prediction to use models from the explainable artificial intelligence field, the evaluation still occurs mainly through predictive performance-based metrics, thus not accounting for the explainability, actionability, and the implications of the results of the models. This paper addresses explainability through the properties interpretability and faithfulness in the field of process outcome prediction. We introduce metrics to analyse these properties along the main dimensions of process data: the event, case, and control flow attributes. This allows for comparing explanations produced by transparent models with explanations generated by (post-hoc) explainability techniques on top of opaque black box models. We utilise thirteen real-life event logs and seven classifiers, encompassing a variety of transparent and non-transparent machine learning and deep learning models, complemented with (post-hoc) explainability techniques. Next, this paper contributes a set of guidelines named X-MOP for obtaining explainable models for outcome prediction, which helps to select the most suitable model by providing insight into how the varying preprocessing, model complexity, and explainability techniques typical in process outcome prediction influence the explainability of the model.