Abstract
Process Model Matching (PMM) aims to automatically identify corresponding activities from two process models that exhibit similar behaviors. Recognizing the diverse applications of process model matching, several techniques have been proposed in the literature. Typically, the effectiveness of these matching techniques has been evaluated using three widely used performance measures, Precision, Recall, and F1 score. In this study, we have established that the values of these three measures for each dataset do not provide deeper insights into the capabilities of the matching techniques. To that end, we have made three significant contributions. Firstly, we have enhanced four benchmark datasets by classifying their corresponding activities into three sub-types. The enhanced datasets can be used for surface-level evaluation, as well as a deeper evaluation of matching techniques. Secondly, we have conducted a systematic search of the literature to identify an extensive set of 27 matching techniques and subsequently proposed a taxonomy for these matching techniques. Finally, we have performed 432 experiments to evaluate the effectiveness of all the matching techniques, and key observations about the effectiveness of the techniques are presented.
Highlights
The conceptual models that depict the workflow of an organization are called business process models [1], [2]
It is abundantly established that these models are valuable assets for organizations, due to the broad range of application areas ranging from documenting software developed requirements to configuring ERP systems [3]
Based-on the results, we have identified the following two interrelated issues regarding the evaluation of Process Model Matching (PMM) techniques
Summary
The conceptual models that depict the workflow of an organization are called business process models [1], [2]. ESTABLISHING THE PROBLEM There are three publicly available datasets that have been widely used to evaluate the effectiveness of PMM techniques [4], [13] These datasets include real-world process models from three different domains: university admission, registration of newborns, and asset management. 2) SURFACE LEVEL EVALUATION Due to the possible diversity in the corresponding pairs, illustrated, the use of a Precision, Recall, or F1 score for a complete dataset merely provides a surface level evaluation of a matching technique That is, it does not provide any deeper insights into the capabilities of PMM techniques. It does not provide any deeper insights into the capabilities of PMM techniques These limitations justify the need for enhancing the benchmark datasets by classifying the corresponding pairs based on the variations in the labels of activities and computing Precision, Recall, and F1 score for each type of the corresponding pair.
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call