Stochastic-aware precision and recall measures for conformance checking in process mining

Abstract

Process mining studies ways to improve real-world processes using historical event data generated by IT systems that support business processes of organisations. Given an event log of an IT system, process discovery algorithms construct a process model representing the processes recorded in the log, while conformance checking techniques quantify how well the discovered model achieves this objective. State-of-the-art discovery and conformance techniques either completely ignore or consider but hide from the users information about the likelihood of process behaviour. That is, the vast majority of the existing process discovery algorithms construct non-stochastic aware process models. Consequently, few conformance checking techniques can assess how well such discovered models describe the relative likelihoods of traces recorded in the log or how well they represent the likelihood of future traces generated by the same system. Note that this is necessary to support process simulation, prediction and recommendation. Furthermore, stochastic information can provide business analysts with further actionable insights on frequent and rare conformance issues. This article presents precision and recall measures based on the notion of entropy of stochastic automata, which are capable of quantifying and, hence, differentiating, between frequent and rare deviations of an event log and a process model that is enriched with the information on the relative likelihoods of traces it describes. An evaluation over several real-world datasets that uses our open-source implementation of the measures demonstrates the feasibility of using our precision and recall measures in industrial settings. Finally, we propose a range of intuitive desired properties that stochastic precision and recall measures should possess, and study our and other existing stochastic-aware conformance measures with respect to these properties.