Optimizing Process Discovery Quality Criteria and Model Measurements using Receiver Operating Characteristic Analysis and Infrequent Inductive Miner

Abstract

Generating process models that reflect close behavioral resemblance to the actual process Standard Operating Procedure (SOP) in process mining can be challenging without taking the four quality criteria of process discovery into account. The four quality criteria, i.e. fitness, precision, generalization, and simplicity, should be well balanced in order to produce proper process models which are aligned to the real-life executions. This paper proposes a method to optimize process discovery quality criteria (PDQC) by implementing different thresholds and analyzing calculation results using Receiver Operating Characteristic (ROC) curve and Infrequent Inductive Miner algorithm. This paper sets up two experiments with different scenarios to measure the calculations of quality criteria and the quality of generated models. The experiments compare two SOPs to the process models discovered by Infrequent Inductive Miner algorithm; hence the SOPs serve as references to determine the generated models quality. The purpose of applying two different scenarios in the experiments is to discover how well the Infrequent Inductive Miner thresholds can produce predictive models under these two different scenarios circumstances. This paper has been successful in predicting the best-fit model in reference to the SOPs by optimizing the four quality criteria of process discovery using ROC thresholds settings and by using infrequent inductive miner algorithm for models generation, and also in improving the accuracy of models measurements. The accuracy rate of the prediction model from Experiment 1 is 83%, while Experiment 2 yields an accuracy rate of 88%. The most optimal threshold settings to generate the best model in this paper are threshold 0.4 in Experiment 1 and threshold 0.5 in Experiment 2.