Predictive Process Monitoring Research Articles

Explaining the impact of design choices on model quality in predictive process monitoring

Explaining the impact of design choices on model quality in predictive process monitoring

Evaluating Early Predictive Performance of Machine Learning Approaches for Engineering Change Schedule – A Case Study Using Predictive Process Monitoring Techniques

By applying machine learning algorithms, predictive business process monitoring (PBPM) techniques provide an opportunity to counteract undesired outcomes of processes. An especially complex variation of business processes is the engineering change (EC) process. Here, failing to adhere to planned implementation dates can have severe impacts on assembly lines, and it is paramount that potential negative cases are identified as early as possible. Current PBPM research, however, has seldomly investigated the predictive performance of machine learning approaches and their applicability at early process steps, let alone for the EC process. In our research, we show that given adequate feature encoding, shallow learners can accurately predict schedule adherence after process initialisation. Based on EC data from an automotive manufacturer, we provide a case sensitive performance overview on algorithm-encoding combinations. For that, three algorithms (XGBoost, Random Forest, LSTM) were combined with four encoding techniques. The encoding techniques used were the two common aggregation-based and index-based last state encoding, and two new combinations of these, which we term advanced aggregation-based and complex aggregation-based encoding. The study indicates that XGBoost-index-encoded approaches outclass regarding average predictive performance, whereas Random-Forest-aggregation-encoded approaches perform better regarding temporal stability due to reduced influence by dynamic features. Our research provides a case-based reasoning approach for deciding on which algorithm-encoding combination and evaluation metrics to apply. In doing so, we provide a blueprint for an early warning and monitoring method within the EC process and other similarly complex processes.

Explainable predictive process monitoring: a user evaluation

Explainability is motivated by the lack of transparency of black-box machine learning approaches, which do not foster trust and acceptance of machine learning algorithms. This also happens in the predictive process monitoring field, where predictions, obtained by applying machine learning techniques, need to be explained to users, so as to gain their trust and acceptance. In this work, we carry on a user evaluation on explanation approaches for predictive process monitoring aiming at investigating whether and how the explanations provided (i) are understandable; (ii) are useful in decision making tasks; (iii) can be further improved for process analysts with different predictive process monitoring expertise levels. The results of the user evaluation show that, although explanation plots are overall understandable and useful for decision making tasks for business process management users — with and without experience in predictive process monitoring — differences exist in the comprehension and usage of different plots, as well as in the way users with different predictive process monitoring expertise understand and use them.

Predictive process monitoring: concepts, challenges, and future research directions

Predictive Process Monitoring (PPM) extends classical process mining techniques by providing predictive models that can be applied at runtime during the execution of a business process, for example, to predict the sequence of the next event(s) in a case, its outcomes, or performance-related aspects such as the remaining processing time. These predictive models go beyond process mining’s inherent descriptive nature that is offered by typical process discovery, conformance checking, and model enhancement techniques. The growing interest in PPM is driven by its additional value proposition, i.e., delivering real-time information regarding the future execution of business process instances, thus allowing for better informed operational decision making and performance analysis. The rapid growth of PPM during the last ten years has left the field lacking a cohesive taxonomy and an explicit recognition of the prevailing challenges. This paper aims at closing this gap, by comprehensively defining PPM using a unified approach to the key terms and by discussing challenges and opportunities in the field. Specifically, we propose three overarching research challenges and nine research directions, which have been validated through a survey with PPM researchers.

LS-ICE: A Load State Intercase Encoding framework for improved predictive monitoring of business processes

Research on developing techniques for predictive process monitoring has generally relied on feature encoding schemes that extract intra-case features from events to make predictions. In doing so, the processing of cases is assumed to be solely influenced by the attributes of the cases themselves. However, cases are not processed in isolation and can be influenced by the processing of other cases or, more generally, the state of the process under investigation. In this work, we propose the LS-ICE (load state intercase encoding) framework for encoding intercase features that enriches events with a depiction of the state of relevant load points in a business process. To assess the benefits of the intercase features generated using the LS-ICE framework, we compare the performance of predictive process monitoring models constructed using the encoded features against baseline models without these features. The models are evaluated for remaining trace and runtime prediction using five real-life event logs. Across the board, a consistent improvement in performance is noted for models that integrate intercase features encoded through the proposed framework, as opposed to baseline models that lack these encoded features.

Counterfactual Explanations in the Big Picture: An Approach for Process Prediction-Driven Job-Shop Scheduling Optimization

In this study, we propose a pioneering framework for generating multi-objective counterfactual explanations in job-shop scheduling contexts, combining predictive process monitoring with advanced mathematical optimization techniques. Using the Non-dominated Sorting Genetic Algorithm II (NSGA-II) for multi-objective optimization, our approach enhances the generation of counterfactual explanations that illuminate potential enhancements at both the operational and systemic levels. Validated with real-world data, our methodology underscores the superiority of NSGA-II in crafting pertinent and actionable counterfactual explanations, surpassing traditional methods in both efficiency and practical relevance. This work advances the domains of explainable artificial intelligence (XAI), predictive process monitoring, and combinatorial optimization, providing an effective tool for improving automated scheduling systems’ clarity, and decision-making capabilities.

Quantifying and explaining machine learning uncertainty in predictive process monitoring: an operations research perspective

AbstractIn the rapidly evolving landscape of manufacturing, the ability to make accurate predictions is crucial for optimizing processes. This study introduces a novel framework that combines predictive uncertainty with explanatory mechanisms to enhance decision-making in complex systems. The approach leverages Quantile Regression Forests for reliable predictive process monitoring and incorporates Shapley Additive Explanations (SHAP) to identify the drivers of predictive uncertainty. This dual-faceted strategy serves as a valuable tool for domain experts engaged in process planning activities. Supported by a real-world case study involving a medium-sized German manufacturing firm, the article validates the model’s effectiveness through rigorous evaluations, including sensitivity analyses and tests for statistical significance. By seamlessly integrating uncertainty quantification with explainable artificial intelligence, this research makes a novel contribution to the evolving discourse on intelligent decision-making in complex systems.

Predictive process model monitoring using long short-term memory networks

The field of predictive process monitoring focuses on case-level models to predict a single specific outcome such as a particular objective, (remaining) time, or next activity/remaining sequence. Recently, a longer-horizon, model-wide approach has been proposed in the form of process model forecasting, which predicts the future state of a whole process model through the forecasting of all activity-to-activity relations at once using time series forecasting.This paper introduces the concept of predictive process model monitoring which sits in the middle of both predictive process monitoring and process model forecasting. Concretely, by modelling a process model as a set of constraints being present between activities over time, we can capture more detailed information between activities compared to process model forecasting, while being compatible with typical predictive process monitoring objectives which are often expressed in the same language as these constraints. To achieve this, Processes-As-Movies (PAM) is introduced, i.e., a novel technique capable of jointly mining and predicting declarative process constraints between activities in various windows of a process’ execution. PAM predicts what declarative rules hold for a trace (objective-based), which also supports the prediction of all constraints together as a process model (model-based). Various recurrent neural network topologies inspired by video analysis tailored to temporal high-dimensional input are used to model the process model evolution with windows as time steps, including encoder–decoder long short-term memory networks, and convolutional long short-term memory networks. Results obtained over real-life event logs show that these topologies are effective in terms of predictive accuracy and precision.

SynBPS: a parametric simulation framework for the generation of event-log data

In the pursuit of ecological validity, current business process simulation methods are calibrated to data from existing processes. This is important for realistic what-if analysis in the context of these processes. However, this is not always the “right tool for the job.” To test hypotheses in the area of predictive process monitoring, it can be more helpful to simulate event-log data from a theoretical process, where all aspects can be manipulated. One example is when assessing the influence of process complexity or variability on the performance of a new prediction method. In this case, the ability to include control variables and systematically change process characteristics is a key to fully understanding their influence. Calibrating a simulation model from observed data alone can in these cases be limiting. This paper proposes a simulation framework, Synthetic Business Process Simulation (SynBPS), a Python library for the generation of event-log data from synthetic processes. Aspects such as process complexity, stability, trace distribution, duration distribution, and case arrivals can be fully controlled by the user. The overall architecture is described in detail, and a demonstration of the framework is presented.

Validation set sampling strategies for predictive process monitoring

Previous studies investigating the efficacy of long short-term memory (LSTM) recurrent neural networks in predictive process monitoring and their ability to capture the underlying process structure have raised concerns about their limited ability to generalize to unseen behavior. Event logs often fail to capture the full spectrum of behavior permitted by the underlying processes. To overcome these challenges, this study introduces innovative validation set sampling strategies based on control-flow variant-based resampling. These strategies have undergone extensive evaluation to assess their impact on hyperparameter selection and early stopping, resulting in notable enhancements to the generalization capabilities of trained LSTM models. In addition, this study expands the experimental framework to enable accurate interpretation of underlying process models and provide valuable insights. By conducting experiments with event logs representing process models of varying complexities, this research elucidates the effectiveness of the proposed validation strategies. Furthermore, the extended framework facilitates investigations into the influence of event log completeness on the learning quality of predictive process models. The novel validation set sampling strategies proposed in this study facilitate the development of more effective and reliable predictive process models, ultimately bolstering generalization capabilities and improving the understanding of underlying process dynamics.

Decision-Making Based on Predictive Process Monitoring of Patient Treatment Processes: A Case Study of Emergency Patients

This paper investigates predictive process monitoring problems in emergency treatment by combining the fields of process management and artificial intelligence. The objective is to predict the next activity and its timestamp in the treatment of emergency patients who have undergone surgery at the gastroenterology or urology surgery units in a hospital in Norway. To achieve this goal, three models were developed using different algorithms, and the best performing model was identified using various performance metrics. The results demonstrate the potential of predictive process monitoring to accurately forecast the outcome of patient treatments. By leveraging the insights gained from predictive process monitoring, hospitals can make more informed decisions. The findings of this study suggest that predictive process monitoring holds significant promise as a tool for improving the efficiency and effectiveness of emergency patient treatment processes. This research has significant implications for the field of decision sciences, particularly regarding resource allocation, reducing waiting times, and improving patient outcomes. The ability to predict the outcomes of patient treatment processes has important implications for hospitals, allowing the streamlining and acceleration of the treatment process. Overall, this study provides a promising framework for predicting patient treatment processes by using the predictive process monitoring method. This could be expanded upon in future research, ultimately leading to improved patient outcomes and better decision-making in healthcare.

Trace encoding in process mining: A survey and benchmarking

Encoding methods are employed across several process mining tasks, including predictive process monitoring, anomalous case detection, trace clustering, etc. These methods are usually performed as preprocessing steps and are responsible for mapping complex event data information into a numerical feature space. Most papers choose existing encoding methods arbitrarily or employ a strategy based on expert domain knowledge. Moreover, existing methods are employed by using their default parameters without evaluating other options. This practice can lead to several drawbacks, such as suboptimal performance and unfair comparisons with the state-of-the-art. Therefore, this work aims at providing a comprehensive survey and benchmark on event log encoding by comparing 27 methods, from different natures, in terms of expressivity, scalability, correlation, and domain agnosticism. To the best of our knowledge, this is the most comprehensive study so far focusing on trace encoding in process mining. It contributes to maturing awareness about the role of trace encoding in process mining pipelines and sheds light on issues, concerns, and future research directions regarding the use of encoding methods to bridge the gap between machine learning models and process mining.

Remaining cycle time prediction: Temporal loss functions and prediction consistency

The usefulness of remaining cycle time models for predictive and prescriptive process monitoring depends not only on the overall accuracy of the predictions but also on their earliness: predictions should be as accurate as possible, as early as possible. To give this criterion more weight in model fitting, the paper evaluates three L1 loss functions with temporal decay. All have the property of increasing the weight of residuals from the early stages of a process relative to residuals from later stages but do so to different degrees. The loss functions are used in LSTM networks for training remaining throughout time models of four different business processes based on publicly available event log data sets. Compared to models trained with unweighted L1 loss, the suggested modifications yield small but significant improvements in earliness on out-of-sample data. Neither the unweighted L1 loss nor the modifications led to models with strictly monotonically decreasing predictions of the remaining time.

Improving Predictive Process Monitoring Through Reachability Graph-Based Masking of Neural Networks

Predicting the next event during process runtime is an objective of interest in predictive process monitoring (PPM). <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Decay replay mining</i> is one of few deep learning-based next event prediction approaches that are built upon process model notations. However, this algorithm does not fully intertwine its neural network with the available process knowledge contained in the process model. This work, which is an extended version of an earlier conference publication, investigates the reachability graphs of underlying Petri net process models for masking the neural network of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">decay replay mining</i> to ultimately increase the quality of next event predictions. A more comprehensive set of experiments is performed to provide robust statistical evidence of the usefulness of the approach and relativizes earlier made claims and hypotheses. In addition, the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">decay replay mining</i> approach is applied with the suggested reachability graph-based masking extension to a healthcare use case of sepsis patients facilitating decision-making for healthcare practitioners. The obtained results further underscore the validity of the masking of neural networks using knowledge contained in the reachability graph of a Petri net process model.

Preserving complex object-centric graph structures to improve machine learning tasks in process mining

Interactions of multiple processes and different objects can be captured using object-centric event data. Object-centric event data represent process executions as event graphs of interacting objects. When applying machine learning techniques to object-centric event data, the event log has to be flattened into sequential process executions used as input for traditional process mining approaches. However, sequentializing the events by flattening removes the graph structure of object-centric event data and, therefore, constitutes an information loss. In this paper, we present a general approach to preserve the graph structures of object-centric event data across machine learning tasks in process mining. We provide two different techniques to preserve these structures depending on the required input format of machine learning techniques: as direct graph encodings or as graph embeddings. We evaluate our contributions by applying three different predictive process monitoring tasks to direct graph encodings, graph embeddings, and flattened event logs. Based on the relative performances, we assess the information contained in the graph structure of object-centric event logs that is, currently, lost in machine learning approaches for traditional process mining. Furthermore, we compare different graph embedding techniques for object-centric event logs to derive recommendations for future research and deployment. To conclude, we assess the improvement that graph embeddings constitute over state-of-the-art approaches that capture object information through specific features. Our contributions improve predictive process monitoring on object-centric event data and quantify the potential performance increases of predictive models. These contributions are especially relevant when looking at real-life applications of predictive process monitoring which are often applied to information systems with relational databases that contain multiple objects and, currently, still enforce flattening.

Utilizing the omnipresent: Incorporating digital documents into predictive process monitoring using deep neural networks

Predictive process monitoring (PPM) allows companies to improve the efficiency of their business processes by predicting aspects such as the process outcome, the next event, or the time until the next event. So far, existing studies have mainly focused on developing novel predictive models while using features solely from event logs. In this study, we aim to go beyond log data and increase the focus of PPM research towards external context information. To this end, we consider digital documents as they are omnipresent in many business processes and their inclusion can often be justified by a business rationale. However, incorporating digital documents into PPM models poses considerable challenges as they present unstructured data that can contain visual and textual cues of future process behavior, while manual feature extraction is generally not feasible. Therefore, we propose an approach that processes digital documents based on automated visual and textual feature extraction methods. Furthermore, we design a tailored integration module which transforms the extracted features from multiple document pages into a fixed-size representation that subsequently serves as input for the predictive models. Our evaluation, based on a real-world dataset of insurance claims from a mid-sized German insurance company, featuring 5131 process instances with 32,058 events and 39,242 document pages, shows that incorporating digital documents improves the performance by significant margins in predicting the damage type, the next event, and the time until the next event. Finally, we analyze how digital documents contribute to the model’s predictions in terms of Shapley additive explanations.

A Direct Data Aware LSTM Neural Network Architecture for Complete Remaining Trace and Runtime Prediction

Developing LSTM neural networks that can accurately predict the future trajectory of ongoing cases and their remaining runtime is an active area of research in predictive process monitoring. In this work a novel complete remaining trace prediction (CRTP) LSTM is proposed. This model is trained to directly predict the complete remaining trace and runtime of cases in contrast to single event prediction as is considered in previously published research on this topic. This makes the CRTP-LSTM robust in terms of utilizing all available attributes of previously observed events for prediction, consequently it can be considered natively data aware. In an extensive experimental assessment the authors show that CRTP-LSTMs consistently outperform other considered approaches for both remaining trace and runtime prediction. Furthermore, the authors show that including all available information contained in previously observed events has a positive impact on the performance of the CRTP-LSTM model. This indicates that valuable information can be extracted from attributes of events in order to make more accurate trace and runtime predictions. This opens up interesting avenues for future research including the incorporation of inter-case features into a modeling setup when predicting the remaining trace and runtime of cases.

DARWIN: An online deep learning approach to handle concept drifts in predictive process monitoring

Predictive process monitoring (PPM) is a specific task under the umbrella of Process Mining that aims to predict several factors of a business process (e.g., next activity prediction) based on the knowledge learned from historical event logs. Despite recent PPM algorithms have gained predictive accuracy using deep learning, they commonly perform an offline analysis of event data assuming that logged processes remain in a steady state over time. However, this is often not the real-world case due to concept drifts. The main goal of this work is to solve the next-activity prediction problem under dynamic conditions of business data streams. To this aim, we propose DARWIN as a novel PPM method that detects concept drifts and adapts a deep neural model to concept drifts. A deep empirical analysis of different factors that may influence the performance of DARWIN in streaming scenarios is provided. Experiments with various benchmark event streams show the effectiveness of the proposed approach.

Global conformance checking measures using shallow representation and deep learning

Conformance checking refers to techniques that can compare normative process behavior, typically captured by process models, and observed process behavior, usually captured in an event log. Recently, machine and deep learning methods have been proposed to tackle multiple problems in process mining, mostly for predictive process monitoring purposes. In this work, we propose two different neural network-based conformance checking techniques, allowing for a fully data-driven approach. Concretely, two approaches for measuring fitness and precision are introduced, using techniques from representation learning and deep learning respectively. The first approach relies on computing the distances between process traces coming from both an event log and a process model by means of vector representations extracted from a shallow neural network trained in a self-supervised setting. The second approach uses a recurrent neural network trained on an event log and a log containing artificially created counterfactuals. The first setup of these methods is tested on a proof-of-concept artificial event log, a plethora of artificially generated process models, and 4 real-life data sets. Our approaches exhibit generally promising and interesting results and in this way provide fully data-driven alternatives to other conformance checking techniques, which are usually model-driven. Moreover, they allow for a possible combination of different purposes, when already using neural networks for process discovery or predictive process monitoring.

Predictive Business Process Monitoring Approach Based on Hierarchical Transformer

Predictive business process monitoring is predicting the next stage of the business process based on the sequence of events that have occurred in the business process instance, which is positive for promoting the rational allocation of resources and the improvement of execution efficiency. There are drawbacks in modeling business process instances, such as conceptual drift phenomenon and long event sequences. Therefore, we propose a hierarchical Transformer-based business process prediction model to improve the performance of the Transformer-based predictive business process monitoring model. We encode the event features using two different encoding methods to obtain the relationship between activities and attributes. A drift detection algorithm is proposed to segment the business process and calculate the correlation between activities and segments by using cross-attention. Furthermore, learnable position encoding is designed to capture the relative position information of subsequences. Finally, the information of different granularity, such as event attributes, event subsequences, and complete instances, is fused by different weights. Experiments were run on seven real event logs for the next activity prediction and remaining time prediction, and the next activity prediction accuracy improved by 6.32% on average, and the mean absolute error of remaining time prediction reduces by 21% on average.