Event Prediction Problem Research Articles

A machine learning case study to predict rare clinical event of interest: imbalanced data, interpretability, and practical considerations

ABSTRACT Accurate prediction of a rare and clinically important event following study treatment has been crucial in drug development. For instance, the rarity of an adverse event is often commensurate with the seriousness of medical consequences, and delayed detection of the rare adverse event can pose significant or even life-threatening health risks to patients. In this machine learning case study, we demonstrate with an example originated from a real clinical trial setting how to define and solve the rare clinical event prediction problem using machine learning in pharmaceutical industry. The unique contributions of this work include the proposal of a six-step investigation framework that facilitates the communication with non-technical stakeholders and the interpretation of the model performance in terms of practical consequences in the context of patient screenings for conducting a future clinical trial. In terms of machine learning methodology, for data splitting into the training and test sets, we adapt the rare-event stratified split approach (from scikit-learn) to further account for group splitting for multiple records of a patient simultaneously. To handle imbalanced data due to rare events in model training, the cost-sensitive learning approach is employed to give more weights to the minor class and the metrics precision together with recall are used to capture prediction performance instead of the raw accuracy rate. Finally, we demonstrate how to apply the state-of-the-art SHAP values to identify important risk factors to improve model interpretability.

A New Method of Time-Series Event Prediction Based on Sequence Labeling

In the existing research on time-series event prediction (TSEP) methods, most of the work is focused on improving the algorithm for classifying subsequence sets (sets composed of multiple adjacent subsequences). However, these prediction methods ignore the timing dependence between the subsequence sets, nor do they capture the mutual transition relationship between events, the prediction effect on a small sample data set is very poor. Meanwhile, the sequence labeling problem is one of the common problems in natural language processing and image segmentation. To solve this problem, this paper proposed a new framework for time-series event prediction, which transforms the event prediction problem into a labeling problem, to better capture the timing relationship between the subsequence sets. Specifically, the framework used a sequence clustering algorithm for the first time to identify representative patterns in the time series, then represented the set of subsequences as a weighted combination of patterns, and used the eXtreme gradient boosting algorithm (XGBoost) for feature selection. After that, the selected pattern feature was used as the input of the long-term short-term memory model (LSTM) to obtain the preliminary prediction value. Furthermore, the fully-linked conditional random field (CRF) was used to smooth and refine the preliminary prediction value to obtain the final prediction result. Finally, the experimental results of event prediction on five real data sets show that the CX-LC method has a certain improvement in prediction accuracy compared with the other six models.

A Predictive Model for Student Employability Using Deep Learning Techniques

Education in the present scenario is outcome based and focuses mainly on the skill sets a student acquires on completion of the studies. The society is increasingly concerned about the quality of programs, international rankings, and placement statistics of HEI (Higher Education Institutions). This study concentrates on how demographic data, scholastic and co-scholastic abilities of students, faculty characteristics, and teaching practices contribute to the student learning. Dataset pertaining to the study were collected from the same institution for which the placement prediction needs to be calculated. The study models the problem as a sequential event prediction problem and employs deep learning techniques. The proposed model extracts data from dataset with 18 attributes. This predictive approach evaluates the performance of lower level and higher order skills and provide the enhancement methods by which a student can be on the path to full-time employment before leaving the campus.

Predicting Hot Events in the Early Period through Bayesian Model for Social Networks

Predicting emerging hot events in an early stage is essential for various applications, including information dissemination mining, ads recommendation and etc. Existing techniques either require a long-term observation over the event or features that are expensive to extract. However, given limited data at the early stage of an emerging event, the temporal features of hot events and non-hot events are not distinctive enough yet. In this work, we introduce BEEP, a Bayesian perspective Early stage Event Prediction model, that tackles this dilemma. We formulate the hot event prediction problem by two Semi-Naive Bayes Classifiers, where we consider both the temporal features and structural features and perform distribution test for the selected features. Theoretical analysis and extensive empirical evaluations on two real datasets demonstrate the effectiveness of our methods.

Multi-Head Spatio-Temporal Attention Mechanism for Urban Anomaly Event Prediction

Timely forecasting the urban anomaly events in advance is of great importance to the city management and planning. However, anomaly event prediction is highly challenging due to the sparseness of data, geographic heterogeneity (e.g., complex spatial correlation, skewed spatial distribution of anomaly events and crowd flows), and the dynamic temporal dependencies. In this study, we propose M-STAP, a novel Multi-head Spatio-Temporal Attention Prediction approach to address the problem of multi-region urban anomaly event prediction. Specifically, M-STAP considers the problem from three main aspects: (1) extracting the spatial characteristics of the anomaly events in different regions, and the spatial correlations between anomaly events and crowd flows; (2) modeling the impacts of crowd flow dynamic of the most relevant regions in each time step on the anomaly events; and (3) employing attention mechanism to analyze the varying impacts of the historical anomaly events on the predicted data. We have conducted extensive experimental studies on the crowd flows and anomaly events data of New York City, Melbourne and Chicago. Our proposed model shows higher accuracy (41.91% improvement on average) in predicting multi-region anomaly events compared with the state-of-the-arts.

POP-ON: Prediction of Process Using One-Way Language Model Based on NLP Approach

In business process management, the monitoring service is an important element that can prevent various problems in advance from before they occur in companies and industries. Execution log is created in an information system that is aware of the enterprise process, which helps predict the process. The ultimate goal of the proposed method is to predict the process following the running process instance and predict events based on previously completed event log data. Companies can flexibly respond to unwanted deviations in their workflow. When solving the next event prediction problem, we use a fully attention-based transformer, which has performed well in recent natural language processing approaches. After recognizing the name attribute of the event in the natural language and predicting the next event, several necessary elements were applied. It is trained using the proposed deep learning model according to specific pre-processing steps. Experiments using various business process log datasets demonstrate the superior performance of the proposed method. The name of the process prediction model we propose is “POP-ON”.

Individual-Level Fatality Prediction of COVID-19 Patients Using AI Methods.

The global covid-19 pandemic puts great pressure on medical resources worldwide and leads healthcare professionals to question which individuals are in imminent need of care. With appropriate data of each patient, hospitals can heuristically predict whether or not a patient requires immediate care. We adopted a deep learning model to predict fatality of individuals tested positive given the patient's underlying health conditions, age, sex, and other factors. As the allocation of resources toward a vulnerable patient could mean the difference between life and death, a fatality prediction model serves as a valuable tool to healthcare workers in prioritizing resources and hospital space. The models adopted were evaluated and refined using the metrics of accuracy, specificity, and sensitivity. After data preprocessing and training, our model is able to predict whether a covid-19 confirmed patient is likely to be dead or not, given their information and disposition. The metrics between the different models are compared. Results indicate that the deep learning model outperforms other machine learning models to solve this rare event prediction problem.

Modeling Marked Temporal Point Process Using Multi-relation Structure RNN

Event sequences with marker and timing information are available in a wide range of domains, from machine log in automatic train supervision systems to information cascades in social networks. Given the historical event sequences, predicting what event will happen next and when it will happen can benefit many useful applications, such as maintenance service schedule for mass rapid transit trains and product advertising in social networks. Temporal point process (TPP) is one effective solution to solve the next event prediction problem due to its capability of capturing the temporal dependence among events. The recent recurrent temporal point process (RTPP) methods exploited recurrent neural network (RNN) to get rid of the parametric form assumption in the density functions of TPP. However, most existing RTPP methods focus only on the temporal dependence among events. In this work, we design a novel multi-relation structure RNN model with a hierarchical attention mechanism to capture not only the conventional temporal dependencies but also the explicit multi-relation topology dependencies. We then propose an RTPP algorithm whose density function conditioned on the event sequence embedding learned from our RNN model for cognitively predict the next event marker and time. The experiments show that our proposed MRS-RMTPP outperforms the state-of-the-art baselines in terms of both event marker prediction and event time prediction on three real-world datasets. The capability of capturing both ontology relation structure and temporal structure in the event sequences is of great importance for the next event marker and time prediction.

Predicting Urban Dispersal Events: A Two-Stage Framework through Deep Survival Analysis on Mobility Data

Urban dispersal events are processes where an unusually large number of people leave the same area in a short period. Early prediction of dispersal events is important in mitigating congestion and safety risks and making better dispatching decisions for taxi and ride-sharing fleets. Existing work mostly focuses on predicting taxi demand in the near future by learning patterns from historical data. However, they fail in case of abnormality because dispersal events with abnormally high demand are non-repetitive and violate common assumptions such as smoothness in demand change over time. Instead, in this paper we argue that dispersal events follow a complex pattern of trips and other related features in the past, which can be used to predict such events. Therefore, we formulate the dispersal event prediction problem as a survival analysis problem. We propose a two-stage framework (DILSA), where a deep learning model combined with survival analysis is developed to predict the probability of a dispersal event and its demand volume. We conduct extensive case studies and experiments on the NYC Yellow taxi dataset from 20142016. Results show that DILSA can predict events in the next 5 hours with F1-score of 0:7 and with average time error of 18 minutes. It is orders of magnitude better than the state-of-the-art deep learning approaches for taxi demand prediction.

A Novel Business Process Prediction Model Using a Deep Learning Method

The ability to proactively monitor business processes is a main competitive differentiator for firms. Process execution logs generated by process aware information systems help to make process specific predictions for enabling a proactive situational awareness. The goal of the proposed approach is to predict the next process event from the completed activities of the running process instance, based on the execution log data from previously completed process instances. By predicting process events, companies can initiate timely interventions to address undesired deviations from the desired workflow. The paper proposes a multi-stage deep learning approach that formulates the next event prediction problem as a classification problem. Following a feature pre-processing stage with n-grams and feature hashing, a deep learning model consisting of an unsupervised pre-training component with stacked autoencoders and a supervised fine-tuning component is applied. Experiments on a variety of business process log datasets show that the multi-stage deep learning approach provides promising results. The study also compared the results to existing deep recurrent neural networks and conventional classification approaches. Furthermore, the paper addresses the identification of suitable hyperparameters for the proposed approach, and the handling of the imbalanced nature of business process event datasets.

Event hazard function learning and survival analysis for tearing mode onset characterization

It is shown that concepts from survival analysis (branch of statistics dealing with various types of time-to-event data) are helpful when trying to quantify and understand the onset of tearing modes in tokamaks. It is argued that a probabilistic event prediction problem should be decomposed into (i) dynamical system evolution and (ii) event hazard function integration. Successful machine learning of a hazard (events per time) function from experimental data is demonstrated. The hazard function exhibits statistical properties that are consistent with expectation. A specific tearing delta-prime proxy is found to not contribute to the likelihood of the hazard function for the present case. Although in this paper the event is the onset of a tearing mode in a particular plasma scenario, these ideas should be equally applicable to disruption events.

Wind Power Ramp Events Prediction with Hybrid Machine Learning Regression Techniques and Reanalysis Data

Wind Power Ramp Events (WPREs) are large fluctuations of wind power in a short time interval, which lead to strong, undesirable variations in the electric power produced by a wind farm. Its accurate prediction is important in the effort of efficiently integrating wind energy in the electric system, without affecting considerably its stability, robustness and resilience. In this paper, we tackle the problem of predicting WPREs by applying Machine Learning (ML) regression techniques. Our approach consists of using variables from atmospheric reanalysis data as predictive inputs for the learning machine, which opens the possibility of hybridizing numerical-physical weather models with ML techniques for WPREs prediction in real systems. Specifically, we have explored the feasibility of a number of state-of-the-art ML regression techniques, such as support vector regression, artificial neural networks (multi-layer perceptrons and extreme learning machines) and Gaussian processes to solve the problem. Furthermore, the ERA-Interim reanalysis from the European Center for Medium-Range Weather Forecasts is the one used in this paper because of its accuracy and high resolution (in both spatial and temporal domains). Aiming at validating the feasibility of our predicting approach, we have carried out an extensive experimental work using real data from three wind farms in Spain, discussing the performance of the different ML regression tested in this wind power ramp event prediction problem.

Learning theory analysis for association rules and sequential event prediction

We present a theoretical analysis for prediction algorithms based on association rules. As part of this analysis, we introduce a problem for which rules are particularly natural, called event prediction. In sequential event prediction, events in a sequence are revealed one by one, and the goal is to determine which event will next be revealed. The training set is a collection of past sequences of events. An example application is to predict which item will next be placed into a customer's online shopping cart, given his/her past purchases. In the context of this problem, algorithms based on association rules have distinct advantages over classical statistical and machine learning methods: they look at correlations based on subsets of co-occurring past events (items a and b imply item c), they can be applied to the sequential event prediction problem in a natural way, they can potentially handle the cold start problem where the training set is small, and they yield interpretable predictions. In this work, we present two algorithms that incorporate association rules. These algorithms can be used both for sequential event prediction and for supervised classification, and they are simple enough that they can possibly be understood by users, customers, patients, managers, etc. We provide generalization guarantees on these algorithms based on algorithmic stability analysis from statistical learning theory. We include a discussion of the strict minimum support threshold often used in association rule mining, and introduce an adjusted confidence measure that provides a weaker minimum support condition that has advantages over the strict minimum support. The paper brings together ideas from statistical learning theory, association rule mining and Bayesian analysis.

A Learning Theory Framework for Association Rules and Sequential Events

We present a framework and generalization analysis for the use of association rules in the setting of supervised learning. We are specifically interested in a sequential event prediction problem where data are revealed one by one, and the goal is to determine what will next be revealed. In the context of this problem, algorithms based on association rules have a distinct advantage over classical statistical and machine learning methods; however, to our knowledge there has not previously been a theoretical foundation established for using association rules in supervised learning. We present two simple algorithms that incorporate association rules. These algorithms can be used both for sequential event prediction and for supervised classification. We provide generalization guarantees on these algorithms based on algorithmic stability analysis from statistical learning theory. We include a discussion of the strict minimum support threshold often used in association rule mining, and introduce an adjusted confidence measure that provides a weaker minimum support condition that has advantages over the strict minimum support. The paper brings together ideas from statistical learning theory, association rule mining and Bayesian analysis.

Predicting weather events using fuzzy rule based system

Discovering and understanding the dynamic phenomena of weather to accurately predict different weather events has been an integral component of scientific investigations worldwide. The weather data, being inherently fuzzy in nature, requires highly complex processing based on human observations, satellite photography, or radar followed by computer simulations. This is further combined with an understanding of the principles of global and local weather dynamics. This paper attempts to solve weather event prediction for Lahore by implementing a fuzzy rule based system. The difficult problem of weather event prediction has been dealt in this paper through two separate experimental settings. In the first experimental setting a smaller dataset consisting of 365 instances with 4 inputs and 8 weather events has been used to develop a fuzzy inference system. In the second experimental setting the developed fuzzy system has been enhanced for a larger dataset consisting of over 2500 data points, having 17 inputs, and 10 weather events. For the later experiments the results of the fuzzy system have been compared with two other models i.e., decision tree (DT) based model and partial least square based regression (PLSR) model. It has been observed in the present study that the performance of the fuzzy system is sensitive to bootstrapping sampling technique that has been used for generating training and test samples for developing the fuzzy, DT and PLSR models. Further the models under consideration have been less sensitive to principal component analysis based dimensionality reduction method.