Conformance Checking Research Articles

Approximate Conformance Checking for Closed-Loop Systems With Neural Network Controllers

Approximate Conformance Checking for Closed-Loop Systems With Neural Network Controllers

Timed alignments with mixed moves

We study conformance checking for timed models, that is, process models that consider both the sequence of events that occur, as well as the timestamps at which each event is recorded. Time-aware process mining is a growing subfield of research, and as tools that seek to discover timing-related properties in processes develop, so does the need for conformance-checking techniques that can tackle time constraints and provide insightful quality measures for time-aware process models. One of the most useful conformance artefacts is the alignment, that is, finding the minimal changes necessary to correct a new observation to conform to a process model. In this paper, we extend the notion of timed distance from a previous work where an edit on an event’s timestamp came in two types, depending on whether or not it would propagate to its successors. Here, these different types of edits have a weighted cost each, and the ratio of their costs is denoted by α. We then solve the purely timed alignment problem in this setting for a large class of these weighted distances (corresponding to α∈{1}∪[2,∞)). For these distances, we provide linear time algorithms for both distance computation and alignment on models with sequential causal processes.

Predictive process monitoring: concepts, challenges, and future research directions

Predictive Process Monitoring (PPM) extends classical process mining techniques by providing predictive models that can be applied at runtime during the execution of a business process, for example, to predict the sequence of the next event(s) in a case, its outcomes, or performance-related aspects such as the remaining processing time. These predictive models go beyond process mining’s inherent descriptive nature that is offered by typical process discovery, conformance checking, and model enhancement techniques. The growing interest in PPM is driven by its additional value proposition, i.e., delivering real-time information regarding the future execution of business process instances, thus allowing for better informed operational decision making and performance analysis. The rapid growth of PPM during the last ten years has left the field lacking a cohesive taxonomy and an explicit recognition of the prevailing challenges. This paper aims at closing this gap, by comprehensively defining PPM using a unified approach to the key terms and by discussing challenges and opportunities in the field. Specifically, we propose three overarching research challenges and nine research directions, which have been validated through a survey with PPM researchers.

Proactive conformance checking: An approach for predicting deviations in business processes

Modern business processes are subject to an increasing number of external and internal regulations. Compliance with these regulations is crucial for the success of organizations. To ensure this compliance, process managers can identify and mitigate deviations between the predefined process behavior and the executed process instances by means of conformance checking techniques. However, these techniques are inherently reactive, meaning that they can only detect deviations after they have occurred. It would be desirable to detect and mitigate deviations before they occur, enabling managers to proactively ensure compliance of running process instances. In this paper, we propose Business Process Deviation Prediction (BPDP), a novel predictive approach that relies on a supervised machine learning model to predict which deviations can be expected in the future of running process instances. BPDP is able to predict individual deviations as well as deviation patterns. Further, it provides the user with a list of potential reasons for predicted deviations. Our evaluation shows that BPDP outperforms existing methods for deviation prediction. Following the idea of action-oriented process mining, BPDP thus enables process managers to prevent deviations in early stages of running process instances.

Timed Genetic Process Mining for Robust Tracking of Processes under Incomplete Event Log Conditions

In process mining, an event log is a structured collection of recorded events that describes the execution of processes within an organization. The completeness of event logs is crucial for ensuring accurate and reliable process models. Incomplete event logs, which can result from system errors, manual data entry mistakes, or irregular operational patterns, undermine the integrity of these models. Addressing this issue is essential for constructing accurate models. This research aims to enhance process model performance and robustness by transforming incomplete event logs into complete ones using a process discovery algorithm. Genetic process mining, a type of process discovery algorithm, is chosen for its ability to evaluate multiple candidate solutions concurrently, effectively recovering missing events and improving log completeness. However, the original form of the genetic process mining algorithm is not optimized for handling incomplete logs, which can result in incorrect models being discovered. To address this limitation, this research proposes a modified approach that incorporates timing information to better manage incomplete logs. By leveraging timing data, the algorithm can infer missing events, leading to process tracking and reconstruction which is more accurate. Experimental results validate the effectiveness of the modified algorithm, showing higher fitness and precision scores, improved process model comparisons, and a good level of coverage without errors. Additionally, several advanced metrics for conformance checking are presented to further validate the process models and event logs discovered by both algorithms.

Learner Assessment System in e-Learning with OBE Approach: Activity Performance, Ability Level and Recommendation

E-learning can lead learners to achieve learning outcomes if it is designed based on several principles. One is applying assessments that motivate and inform ability levels. In Outcome-based Education (OBE), assessment is integral to the system. However, e-learning has limitations in providing assessment instruments according to needs, such as assessing complex and detailed aspects and accommodating a variety of numerical and linguistic assessment data. Moreover, the presence and involvement of learners affect their performance and learning outcomes. This study proposes a learner assessment system in e-learning with the OBE approach, including learning design, activity performance analysis, ability level determination, and recommendations. This system adds the e-rubric to e-learning to overcome instrument limitations and accommodate comprehensive assessments. Various numerical and linguistic assessment data are unified using 2-tuple fuzzy linguistics, producing ability levels as two tuples. Performance analysis was based on event log data using descriptive statistical technique and alignment-based conformance checking, from frequency, time, and sequence of activity objects, resulting in five activity performance variables. The performance value of each variable is converted into High, Medium, or Low levels. The ability and performance levels are processed using rule-based methods to produce recommendations for learning stages and activity performance directions. The results of this research can be used as input for academic stakeholders and online learning providers and potentially be applied to the advancement of e-learning in higher education. Doi: 10.28991/HIJ-2024-05-03-03 Full Text: PDF

A Data-Driven Approach to Discovering Process Choreography

Implementing approaches based on process mining in inter-organizational collaboration environments presents challenges related to the granularity of event logs, the privacy and autonomy of business processes, and the alignment of event data generated in inter-organizational business process (IOBP) execution. Therefore, this paper proposes a complete and modular data-driven approach that implements natural language processing techniques, text similarity, and process mining techniques (discovery and conformance checking) through a set of methods and formal rules that enable analysis of the data contained in the event logs and the intra-organizational process models of the participants in the collaboration, to identify patterns that allow the discovery of the process choreography. The approach enables merging the event logs of the inter-organizational collaboration participants from the identified message interactions, enabling the automatic construction of an IOBP model. The proposed approach was evaluated using four real-life and two artificial event logs. In discovering the choreography process, average values of 0.86, 0.89, and 0.86 were obtained for relationship precision, relation recall, and relationship F-score metrics. In evaluating the quality of the built IOBP models, values of 0.95 and 1.00 were achieved for the precision and recall metrics, respectively. The performance obtained in the different scenarios is encouraging, demonstrating the ability of the approach to discover the process choreography and the construction of business process models in inter-organizational environments.

Enjoy the silence: Analysis of stochastic Petri nets with silent transitions

Capturing stochastic behaviour in business and work processes is essential to quantitatively understand how nondeterminism is resolved when taking decisions within the process. This is of special interest in process mining, where event data tracking the actual execution of the process are related to process models, and can then provide insights on frequencies and probabilities. Variants of stochastic Petri nets provide a natural formal basis to represent stochastic behaviour and support different data-driven and model-driven analysis tasks in this spectrum. However, when capturing business processes, such nets inherently need a labelling that maps between transitions and activities. In many state of the art process mining techniques, this labelling is not 1-on-1, leading to unlabelled transitions and activities represented by multiple transitions. At the same time, they have to be analysed in a finite-trace semantics, matching the fact that each process execution consists of finitely many steps. These two aspects impede the direct application of existing techniques for stochastic Petri nets, calling for a novel characterisation that incorporates labels and silent transitions in a finite-trace semantics. In this article, we provide such a characterisation starting from generalised stochastic Petri nets and obtaining the framework of labelled stochastic processes (LSPs). On top of this framework, we introduce different key analysis tasks on the traces of LSPs and their probabilities. We show that all such analysis tasks can be solved analytically, in particular reducing them to a single method that combines automata-based techniques to single out the behaviour of interest within an LSP, with techniques based on absorbing Markov chains to reason on their probabilities. Finally, we demonstrate the significance of how our approach in the context of stochastic conformance checking, illustrating practical feasibility through a proof-of-concept implementation and its application to different datasets.

Challenges in AI-supported Process Analysis in the Italian Judicial System: what After Digitalization?

In this commentary article, we outline research challenges and possible directions for the potential applications of AI in the judicial domain by specifically considering process analysis in the Italian context. Applying AI to process analysis poses several challenges, including information extraction from legacy information systems and analysis of legal documents, process modeling with a particular emphasis on temporal analysis, real-time process monitoring, conformance and compliance checking, predictive techniques for accurate predictions, and analysis of judges’ workload. Solutions to these challenges include methods and tools for data identification and collection, innovative approaches to process modeling, reactive techniques for real-time monitoring, conformance checking with explainability, language models adapted to specific domains, and the identification of suitable indicators for the analysis of case handling efficiency and case classification.

Sampling business process event logs with guarantees

SummaryEvent log sampling has emerged as a key research focus in the field of process mining, aiming to enhance the efficiency of various process mining tasks, including model discovery, conformance checking, and process prediction. However, current log sampling techniques often fail to ensure high‐quality sample logs. This paper introduces a novel framework to support efficient event log sampling without compromising the quality of the sample log compared to the original one. The approach revolves around the consideration of directly‐follows relation (DFR) among business tasks as the fundamental behavior unit of an event log. By ensuring the DFR equivalence between the original and sample logs, the proposed technique addresses the challenge of sample log quality from the model discovery point of view. The framework is instantiated by seven distinct sampling strategies each has its own specialty and is fully implemented in the open‐source process mining tool platform ProM. To validate its effectiveness, we conducted a comprehensive experimental evaluation using 12 publicly available real‐life event logs against state‐of‐the‐art sampling techniques. The results clearly demonstrate that our technique significantly improves model discovery efficiency while upholding high quality of the discovered models.

Comparing Conformance Checking for Decision Mining: An Axiomatic Approach

Comparing Conformance Checking for Decision Mining: An Axiomatic Approach

Re-ordered fuzzy conformance checking for uncertain clinical records

Modern hospitals implement clinical pathways to standardize patients’ treatments. Conformance checking techniques provide an automated tool to assess whether the actual executions of clinical processes comply with the corresponding clinical pathways. However, clinical processes are typically characterized by a high degree of uncertainty, both in their execution and recording. This paper focuses on uncertainty related to logging clinical processes. The logging of the activities executed during a clinical process in the hospital information system is often performed manually by the involved actors (e.g., the nurses). However, such logging can occur at a different time than the actual execution time, which hampers the reliability of the diagnostics provided by conformance checking techniques. To address this issue, we propose a novel conformance checking algorithm that leverages principles of fuzzy set theory to incorporate experts’ knowledge when generating conformance diagnostics. We exploit this knowledge to define a fuzzy tolerance in a time window, which is then used to assess the magnitude of timestamp violations of the recorded activities when evaluating the overall process execution compliance. Experiments conducted on a real-life case study in a Dutch hospital show that the proposed method obtains more accurate diagnostics than the state-of-the-art approaches. We also consider how our diagnostics can be used to stimulate discussion with domain experts on possible strategies to mitigate logging uncertainty in the clinical practice.

REACH: Researching Efficient Alignment-based Conformance Checking

Conformance checking techniques compare how a process is supposed to be executed according to a model with how it is executed in reality according to an event log. Alignment-based approaches are the most successful solutions for conformance checking. Optimal alignments are a way of finding the best match between the real and the modeled behavior and identifying the differences. However, finding these optimal alignments is a challenging task, especially for complex cases where the log and the model have many events and paths. The difficulty lies in the computational complexity required to find these alignments. To address this problem, we propose an efficient algorithm named REACH based on the A* search algorithm. The core components of the proposal are the use of a partial reachability graph for faster execution of process models for alignment computation and a set of optimization techniques for reducing the number of states explored by the A* algorithm. These improve performance by both reducing the required computation time per state and the number of states to process respectively. To evaluate the performance and scalability, we conducted tests using 227 pairs of logs and models, comparing the results obtained with those from 10 state-of-the-art approaches. Results show that REACH outperforms the other proposals in runtimes, and even aligns logs and models that no other algorithm is able to align.

Determination of the KIC Fracture Toughness of the X210Cr12 High-Strength Material

Purpose: It is an important problem that the machines become unusable due to the deformation of the machine elements produced before the planned time. In this study, it was aimed to determine the KIC value of X210Cr12, which is a high strength material. In this way, more accurate load values can be used in the design.Methods: In the experimental study to determine the KIC value, the sample geometry, the crack depth, the load-dependent parameters and calculations and some conformity checks were carried out. Experimental system; KIC consists of 3-point flexure specimens, a press, an electronic circuit capable of detecting the change in load crack opening, and a logger.Findings: In the study, the loads were determined from experimental graphs. Subsequently, load-crack opening values were determined. With these data, the KIC values were calculated as 719,7 and 839,7 MPa.mm1/2 as the minumum and maximum values, respectively.Conclusion: The experimental graphs are in the form of curves that break abruptly with unstable crack propagation without showing plastic deformation. There is no study in the literature on the KIC value of X210Cr12 and it has been determined that it has a low KIC value compared to the high-strength steels studied. It is valuable to determine the KIC value, as fracture problems may occur in designs prepared with the material. The results of the study are data at room temperature.

DOMICO: Checking conformance between domain models and implementations

AbstractAs a predominant design method for microsservices architecture (MSA), domain‐driven design (DDD) utilizes a series of standard patterns in both models and implementations to effectively support the design of architectural elements. However, an implementation may deviate from its original domain model that uses certain patterns. The deviation between a domain model and its implementation is a type of architectural drift, which needs to be detected promptly. This paper proposes an approach, namely DOMICO, to check the conformance between the domain model and its implementation, by which the conformance is formalized by defining eight common structural patterns of domain modeling and their representations in both models and the corresponding source code. Based on the formalization, our approach can not only identify the discrepancies (e.g., divergence, absence, and modification) with respect to pattern elements, but also detect possible violations of 24 compliance rules imposed by the patterns. To validate DOMICO, we performed a case study to investigate its use in a supply chain project and its performance. The results show that DOMICO can accurately identify 100% inconsistency issues in the cases examined. As the first conformance checking approach for DDD, DOMICO can be integrated into the regular domain modeling process and help ensure the conformity of microservice implementations to models.

Real-world data analysis of distributed PV and battery energy storage system curtailment in low voltage networks

Curtailment of distributed photovoltaic (PV) and battery energy storage systems will have significant implications for power system transition around the world. Australia offers a unique case study as it has highest installation rate of distributed-PV and growing fleet of battery energy storage systems. Distributed-PV and battery inverters in Australia are required to exhibit voltage-responsive power-quality response modes to prevent excessive voltage rise caused by increasing day-time energy exports, but these modes can curtail power output and limit the value that can be gained from the renewable energy assets. For the first time in Australia, this paper studies all three inverter power-quality response modes and develops novel algorithms to assess curtailment. The algorithms are applied to a real-world dataset of 1-sec distributed-PV and battery power measurements from 1300 residential households; therefore, data resolution is higher and sample size is larger than previous related studies. Using these datasets, the study compares the experienced curtailment between distributed-PV only sites and distributed-PV coupled with battery sites for the first time. The study investigates and quantifies the energy loss due to curtailment within the South Australia region with over 40% distributed-PV penetration which has the highest uptake in the world. The results show that average curtailment loss was 1.5% for distributed-PV only sites and 0.2% for distributed-PV coupled with battery sites, however, some sites were heavily impacted losing up to 25% of generation. The reasons for the larger curtailment loss were identified as wiring and network connection faults and high voltage set-points at households' respective local substations. Other potential reason for the heavily impacted sites is being located towards the end of the low voltage feeder, however this couldn't be verified due to lack of spatial data. The large variation in curtailed generation across sites raises concerns for system owners and suggests further inquiry into the power-quality response mode design and regulations. The study also examines inverter conformance using real-world operational data for all three power quality response modes for the first time, and the results showed a low rate of conformance according to the Australian Standards and there were various discrepancies in the execution of these modes which raises concerns considering the increasing importance of distributed-PV and battery inverter fleet in supporting network operations and security. The results therefore warrant further investigation into the reasons behind these discrepancies and devising appropriate mechanisms to tighten inverter conformance checks.

Multi-perspective conformance checking of uncertain process traces: An SMT-based approach

Conformance checking, one of the central tasks in process mining, compares the expected behavior described by a reference process model to the actual behavior recorded in an event log, with the goal of detecting deviations. Traditionally, it is assumed that the log provides a faithful and complete digital footprint of reality. However, assuming perfect logs is often unrealistic: real-life logs typically suffer from data quality issues, exposing uncertainty in their events, timestamps, and data attributes. We attack this problem by introducing a comprehensive framework for multi-perspective conformance checking dealing with uncertainty along three perspectives: control-flow, time, and data. From the modeling point of view, we consider process models formalized as Petri nets operating over data variables, and event logs presenting uncertainty at the event- and attribute-level. We cast conformance checking as an alignment problem, extending the traditional notions of alignment and cost function to deal with uncertainty along the three aforementioned perspectives. From the operational point of view, we show how (optimal) alignments can be computed through well-established automated reasoning techniques from Satisfiability Modulo Theories (SMT). Specifically, we show how previous results on data-aware SMT-based conformance checking can be lifted to this more sophisticated setting, obtaining a flexible framework that can seamlessly handle different variants of the problem. We formally prove correctness of our approach and implement it in the conformance checker cocomot. Finally, we perform a thorough experimental evaluation on synthetic and real-life logs, demonstrating the overall promising performance of our framework.

Explainable conformance checking: Understanding patterns of anomalous behavior

Anomaly detection in the execution of business processes in the organizations has a high level of complexity due to the consideration of various process perspectives and their constraints, business rules, privacy policies, and regulations. Furthermore, only detection of anomalies is not sufficient. There is a crucial need for the results of anomaly detection to be explainable and interpretable, enabling users to adapt the decision making process and handle the detected anomalies. The work presented in this paper aims to provide business process owners with human-interpretable explanations for patterns of deviations found between process models and event data recorded during the execution of business processes, while traditional conformance checking methods only report low-level deviations found.First, by introducing an automated approach for multi-perspective conformance checking and anomaly detection in business process executions, we extract expected and unexpected behavior from event logs and identify patterns in deviations. Finally, by focusing on identifying the context involved in patterns of unexpected behavior, our approach facilitates the interpretations of detected patterns. The approach has been implemented in the open source ProM framework and its applicability is evaluated through a real-life dataset from a financial organization. The experiment not only shows that we can automatically detect more complex anomalies such as spurious data processing and misusage of authorizations, but also that we can explain these deviations in context.

Data-Aware Declarative Process Mining with SAT

Process Mining is a family of techniques for analyzing business process execution data recorded in event logs. Process models can be obtained as output of automated process discovery techniques or can be used as input of techniques for conformance checking or model enhancement. In Declarative Process Mining, process models are represented as sets of temporal constraints (instead of procedural descriptions where all control-flow details are explicitly modeled). An open research direction in Declarative Process Mining is whether multi-perspective specifications can be supported, i.e., specifications that not only describe the process behavior from the control-flow point of view, but also from other perspectives like data or time. In this article, we address this question by considering SAT (Propositional Satisfiability Problem) as a solving technology for a number of classical problems in Declarative Process Mining, namely, log generation, conformance checking, and temporal query checking. To do so, we first express each problem as a suitable FO (First-Order) theory whose bounded models represent solutions to the problem, and then find a bounded model of such theory by compilation into SAT.

Data-aware conformance checking with SMT

Conformance checking is a key process mining task to confront the normative behavior imposed by a process model with the actual behavior recorded in a log. While this problem has been extensively studied for pure control-flow processes, data-aware conformance checking has received comparatively little attention. In this paper, we tackle the conformance checking problem for the challenging scenario of processes that combine data and control-flow dimensions. Concretely, we adopt the formalism of data Petri nets (DPNs) and show how solid, well-established automated reasoning techniques from the area of Satisfiability Modulo Theories (SMT) can be effectively harnessed to compute conformance metrics and optimal data-aware alignments. To this end, we introduce the CoCoMoT (Computing Conformance Modulo Theories) framework, with a fourfold contribution. First, we show how SMT allows to leverage SAT-based encodings for the pure control-flow setting to the data-aware case. Second, we introduce a novel preprocessing technique based on a notion of property-preserving clustering, to speed up the computation of conformance checking outputs. Third, we show how our approach extends seamlessly to the more comprehensive conformance checking artifacts of multi- and anti-alignments. Fourth, we describe a proof-of-concept implementation based on state-of-the-art SMT solvers, and report on experiments. Finally, we discuss how CoCoMoT directly lends itself to further process mining tasks like log analysis by clustering and model repair, and the use of SMT facilitates the support of even richer multi-perspective models, where, for example, more expressive DPN guards languages are considered or generic datatypes (other than integers or reals) are employed.