Rare Events Data Research Articles

A Comprehensive Survey on Rare Event Prediction

Rare event prediction involves identifying and forecasting events with a low probability using machine learning (ML) and data analysis. Due to the imbalanced data distributions, where the frequency of common events vastly outweighs that of rare events, it requires using specialized methods within each step of the ML pipeline, i.e., from data processing to algorithms to evaluation protocols. Predicting the occurrences of rare events is important for real-world applications, such as Industry 4.0, and is an active research area in statistical and ML. This paper comprehensively reviews the current approaches for rare event prediction along four dimensions: rare event data, data processing, algorithmic approaches, and evaluation approaches. Specifically, we consider 73 datasets from different modalities (i.e., numerical, image, text, and audio), four major categories of data processing, five major algorithmic groupings, and two broader evaluation approaches. This paper aims to identify gaps in the current literature and highlight the challenges of predicting rare events. It also suggests potential research directions, which can help guide practitioners and researchers.

Coding with the machines: machine-assisted coding of rare event data.

While machine coding of data has dramatically advanced in recent years, the literature raises significant concerns about validation of LLM classification showing, for example, that reliability varies greatly by prompt and temperature tuning, across subject areas and tasks-especially in "zero-shot" applications. This paper contributes to the discussion of validation in several different ways. To test the relative performance of supervised and semi-supervised algorithms when coding political data, we compare three models' performances to each other over multiple iterations for each model and to trained expert coding of data. We also examine changes in performance resulting from prompt engineering and pre-processing of source data. To ameliorate concerns regarding LLM's pre-training on test data, we assess performance by updating an existing dataset beyond what is publicly available. Overall, we find that only GPT-4 approaches trained expert coders when coding contexts familiar to human coders and codes more consistently across contexts. We conclude by discussing some benefits and drawbacks of machine coding moving forward.

Comparing various Bayesian random‐effects models for pooling randomized controlled trials with rare events

AbstractThe meta‐analysis of rare events presents unique methodological challenges owing to the small number of events. Bayesian methods are often used to combine rare events data to inform decision‐making, as they can incorporate prior information and handle studies with zero events without the need for continuity corrections. However, the comparative performances of different Bayesian models in pooling rare events data are not well understood. We conducted a simulation to compare the statistical properties of four parameterizations based on the binomial‐normal hierarchical model, using two different priors for the treatment effect: weakly informative prior (WIP) and non‐informative prior (NIP), pooling randomized controlled trials with rare events using the odds ratio metric. We also considered the beta‐binomial model proposed by Kuss and the random intercept and slope generalized linear mixed models. The simulation scenarios varied based on the treatment effect, sample size ratio between the treatment and control arms, and level of heterogeneity. Performance was evaluated using median bias, root mean square error, median width of 95% credible or confidence intervals, coverage, Type I error, and empirical power. Two reviews are used to illustrate these methods. The results demonstrate that the WIP outperforms the NIP within the same model structure. Among the compared models, the model that included the treatment effect parameter in the risk model for the control arm did not perform well. Our findings confirm that rare events meta‐analysis faces the challenge of being underpowered, highlighting the importance of reporting the power of results in empirical studies.

Bayesian typhoon precipitation prediction with a mixture of ensemble forecast-based and historical event-based prediction functions

Statistical post-processing has been widely adopted in probabilistic forecasting, especially when a predictive distribution of weather variables is of interest. Among the available post-processing methods, a common tool is the ensemble Bayesian model averaging. Challenges arise when it is applied to heavy rain prediction, including the precipitation produced by tropical cyclones. The issues are related to the rare extreme event and limited available data for model training. To address this, we propose in this research a Bayesian mixture model, Bmix, based on information from forecast members and historical tropical cyclones, rather than solely on immediately observed precipitation data. This approach provides a grid-specific Bayesian predictive distribution and an easy-to-derive probabilistic forecast with posterior samples generated from the Markov chain Monte Carlo algorithm. This post-processing procedure can be executed on each grid simultaneously, reducing the computational time significantly. In addition, a categorized Bmix (C.Bmix) is implemented to accommodate different scenarios such as heavy or torrential rain in applications. The proposed approaches are demonstrated on the 24-hour accumulated precipitation forecast of a category 4-equivalent super typhoon Dujuan landing in Taiwan in 2015; while two others, Matmo and Soudelor, are adopted to provide historical information. Over the Taiwan main island, a total of 1282 grids are considered, each with a resolution of 5 km and 20 ensemble member forecasts from the Weather Research and Forecasting Ensemble Prediction System (WEPS) at a 6-hour interval during the typhoon period. The analyses show that the proposed mixture approaches present larger percentages of positive CRPSS values than traditional BMA. For instance, at the time when Dujuan made landfall, the percentages are 92.2 % for Bmix, 95.2 % for C.Bmix, and 15.4 % for BMA when the true precipitation amount exceeds 200 mm/24 h. The proposed methods outperform in the probability prediction and the overall pattern of precipitation.

A NEW APPROACH TO DETERMINE THE INFLUENCE OF WEATHER CONDITIONS ON FOREST FIRE RISK IN THE MEDITERRANEAN REGION OF TÜRKİYE

The risk of forest fires is a major problem in Türkiye's Mediterranean region and has a significant impact on ecosystems and atmospheric conditions. Throughout the previous century, a significant portion of Türkiye's Mediterranean Region has been destroyed by forest fires. This study aims to determine the meteorological covariates, such as relative humidity, maximum temperature, and wind speed, that affect forest fires. We classified forest fires into two groups. The first group (F1) refers to small forest fires, with burned forest areas of less than 10 hectares. The second group (F2), representing rare events, corresponds to burned areas of more than 10 hectares. The data is composed of binary values (F1=0 and F2=1) taken between the years 2015-2019 from different locations in the Mediterranean Region of Türkiye. For binary data modeling, the ordinary logistic regression (LR) has been frequently used. However, such a method tends to give biased results when using rare event data. Therefore, we employed three different modeling techniques specifically designed for rare event data. According to the results obtained from the best model, Firth's Logistic Regression (FLR), wind speed, and maximum temperature are found to be statistically significant variables in the occurrence of forest fires greater than 10 hectares.

Predictive factors of atopic-like dermatitis induced by IL-17A inhibitors in patients with psoriasis: A 2-year follow-up study.

Atopic-like dermatitis (ALD) is a common side effect of interleukin-17A (IL-17A) inhibitors. To determine the prevalence, risk factors, outcomes and treatment of ALD in a cohort of psoriasis patients treated with IL-17A inhibitors. This retrospective study included 226 psoriasis patients treated with an IL-17A inhibitor in our dermatology department between July 2020 and July 2022. The patients were reviewed over 2 years. A logistic regression model in rare events data (relogit) was used to predict the risk factors for ALD. Of the 226 patients, 14 had ALD. Data including age, body mass index, IL-17A inhibitor use, personal and family history of atopic disease, pet ownership history, and immunoglobulin E (IgE) levels were analysed using the relogit regression model. It indicated a personal history of atopic disease (odd ratio [OR] 27.830, 95% confidence interval [CI] 3.801-203.770; p = 0.001) and elevated IgE levels (OR 5.867, 95% CI 1.131-30.434; p = 0.035) as independent predictors of incident ALD. In one patient, anti-IL-17A therapy was discontinued, and treatment was switched to tofacitinib. Thirteen patients who continued with IL-17A inhibitor were treated with topical therapy and/or antihistamines, and their ALD was partially or completely resolved. In this study, the incidence rate of ALD was 6.19%. Elevated IgE levels and a personal history of atopic disease were found to be the risk factors for ALD. Our study findings suggest that treatment should be provided based on the severity of psoriasis and incident ALD. Prior to treatment, psoriasis patients who have the risk factors for ALD should be informed of the possible development of ALD, and alternative psoriatic therapeutic options should be considered if severe ALD develops.

Improving performance of hurdle models using rare-event weighted logistic regression: an application to maternal mortality data.

In this paper, performance of hurdle models in rare events data is improved by modifying their binary component. The rare-event weighted logistic regression model is adopted in place of logistic regression to deal with class imbalance due to rare events. Poisson Hurdle Rare Event Weighted Logistic Regression (REWLR) and Negative Binomial Hurdle (NBH) REWLR are developed as two-part models which use the REWLR model to estimate the probability of a positive count and a Poisson or NB zero-truncated count model to estimate non-zero counts. This research aimed to develop and assess the performance of the Poisson and Negative Binomial (NB) Hurdle Rare Event Weighted Logistic Regression (REWLR) models, applied to simulated data with various degrees of zero inflation and to Nairobi county's maternal mortality data. The study data on maternal mortality were pulled from JPHES. The data contain the number of maternal deaths, which is the outcome variable, and other obstetric and demographic factors recorded in MNCH facilities in Nairobi between October 2021 and January 2022. The models were also fit and evaluated based on simulated data with varying degrees of zero inflation. The obtained results are numerically validated and then discussed from both the mathematical and the maternal mortality perspective. Numerical simulations are also presented to give a more complete representation of the model dynamics. Results obtained suggest that NB Hurdle REWLR is the best performing model for zero inflated count data due to rare events.

Long-term mental health consequences of female- versus male-perpetrated child sexual abuse

BackgroundResearch on child sexual abuse increasingly focuses on sexually offending females; however, there is a lack of research that focuses on the individuals being affected. Studies have suggested that the consequences for those affected by sexually offending males and females are comparable. ObjectiveThe aim is to compare mental health consequences of sexual abuse perpetrated by women versus men in quantity and type. Participants and settingData was anonymously retrieved from the German-wide contact point “help line sexual abuse” from 2016 to 2021. Details of abuse cases, gender of the offending individuals, and reported mental disorders of the person affected were analyzed. The sample consisted of N = 3351 callers with experiences of child sexual abuse. MethodThe relationship between gender of the perpetrating person and mental disorders of the victimized person was computed using logistic regression models. To account for rare events data, Firth's logistics regression model was used. ResultsThe consequences were similar in quantity, albeit different in type. Callers with experiences of female-perpetrated abuse were more likely to report suicidality, non-suicidal self-injury behavior, personality disorders, dissociative identity disorders, alcohol/ drug addiction, and schizophrenia, while people with experiences of male-perpetrated abuse rather reported post-traumatic stress disorder, affective disorders, anxiety disorders, dissociative disorders, eating disorders, externalized disorders, and psychosomatic disorders. ConclusionsThe differences may be due to stigmatization leading to dysfunctional coping mechanisms. Societal gender stereotypes must be reduced, especially within the professional helping system to ensure support for people who have experienced sexual abuse, regardless of gender.

Identification of high-risk roadway segments for wrong-way driving crash using rare event modeling and data augmentation techniques

Wrong-Way Driving (WWD) crashes are relatively rare but more likely to produce fatalities and severe injuries than other crashes. WWD crash segment prediction task is challenging due to its rare nature, and very few roadway segments experience WWD events. WWD crashes involve complex interactions among roadway geometry, vehicle, environment, and drivers, and the effect of these complex interactions is not always observable and measurable. This study applied two advanced Machine Learning (ML) models to overcome the imbalanced dataset problem and identified local and global factors contributing to WWD crash segments. Five years (2015–2019) of WWD crash data from Florida state were used in this study for WWD model development. The first modeling approach applied four different hybrid data augmentation techniques to the training dataset before applying the XGBoost classification algorithm. In the second model, a rare event modeling approach using the Autoencoder-based anomaly detection method was applied to the original data to identify WWD roadway segments. A third model was applied based on the statistical method to compare the performance of ML models in predicting the WWD segments. The performance comparison of the adopted models showed that the XGBoost model with the Adaptive Synthetic Sampling (ADASYN) method performed best in terms of precision and recall values compared to the autoencoder-based anomaly detection method. The best-performing model was used for the feature analysis with an interpretable machine-learning technique. The SHapley Additive exPlanations (SHAP) values showed that high-intensity developed land use, length of roadway, log of Annual Average Daily traffic (AADT), and lane width were positively associated with WWD roadway segments. The results of this study can be used to deploy WWD countermeasures effectively.

AutoScore-Imbalance: An interpretable machine learning tool for development of clinical scores with rare events data

BackgroundMedical decision-making impacts both individual and public health. Clinical scores are commonly used among various decision-making models to determine the degree of disease deterioration at the bedside. AutoScore was proposed as a useful clinical score generator based on machine learning and a generalized linear model. However, its current framework still leaves room for improvement when addressing unbalanced data of rare events. MethodsUsing machine intelligence approaches, we developed AutoScore-Imbalance, which comprises three components: training dataset optimization, sample weight optimization, and adjusted AutoScore. Baseline techniques for performance comparison included the original AutoScore, full logistic regression, stepwise logistic regression, least absolute shrinkage and selection operator (LASSO), full random forest, and random forest with a reduced number of variables. These models were evaluated based on their area under the curve (AUC) in the receiver operating characteristic analysis and balanced accuracy (i.e., mean value of sensitivity and specificity). By utilizing a publicly accessible dataset from Beth Israel Deaconess Medical Center, we assessed the proposed model and baseline approaches to predict inpatient mortality. ResultsAutoScore-Imbalance outperformed baselines in terms of AUC and balanced accuracy. The nine-variable AutoScore-Imbalance sub-model achieved the highest AUC of 0.786 (0.732–0.839), while the eleven-variable original AutoScore obtained an AUC of 0.723 (0.663–0.783), and the logistic regression with 21 variables obtained an AUC of 0.743 (0.685–0.801). The AutoScore-Imbalance sub-model (using a down-sampling algorithm) yielded an AUC of 0.771 (0.718–0.823) with only five variables, demonstrating a good balance between performance and variable sparsity. Furthermore, AutoScore-Imbalance obtained the highest balanced accuracy of 0.757 (0.702–0.805), compared to 0.698 (0.643–0.753) by the original AutoScore and the maximum of 0.720 (0.664–0.769) by other baseline models. ConclusionsWe have developed an interpretable tool to handle clinical data imbalance, presented its structure, and demonstrated its superiority over baselines. The AutoScore-Imbalance tool can be applied to highly unbalanced datasets to gain further insight into rare medical events and facilitate real-world clinical decision-making.

Whose trade follows the flag? Institutional constraints and economic responses to bilateral relations

This study revisits the association between bilateral relations and trade based on rare-event data from Integrated Data for Event Analysis (IDEA). Our results suggest that a country imports more from another if the two countries are friendlier. We further argue that states face two constraints when attempting to manipulate trade. First, they are constrained by domestic institutions such as elections and congress. Second, they are constrained by international institutions such as the World Trade Organization (WTO). Our results show that the imports of authoritarian countries follow the flag of politics, but democratic countries’ imports are less likely to be affected by bilateral relations. Moreover, WTO membership can gradually restrict democratic states from intervening on imports but has little impact on authoritarian governments.

A comparison of confidence distribution approaches for rare event meta-analysis.

Meta-analysis of rare event data has recently received increasing attention due to the challenging issues rare events pose to traditional meta-analytic methods. One specific way to combine information and analyze rare event meta-analysis data utilizes confidence distributions (CDs). While several CD methods exist, no comparisons have been made to determine which method is best suited for homogeneous or heterogeneous meta-analyses with rare events. In this article, we review several CD methods: Fisher's classic P-value combination method, one that combines P-value functions, another that combines confidence intervals, and one that combines confidence log-likelihood functions. We compare these CD approaches, and we propose and compare variations of these methods to determine which method produces reliable results for homogeneous or heterogeneous rare event meta-analyses. We find that for homogeneous rare event data, most CD methods perform very well. On the other hand, for heterogeneous rare event data, there is a clear split in performance between some CD methods, with some performing very poorly and others performing reasonably well.

Comparison of Different Estimation Approaches in Rare Events Data

In social science researches, there may be cases where a category of the dependent variable is seen hundred times less (more) than the other category. Events like wars, mass migrations or coups in social sciences; an event of interest in binary variable(s) may have very low prevalence, resulting in low or even zero cell counts in one or two cells in the 2X2 tables of two factors. In this case, independent variable predict the dependent variable perfectly or almost perfectly, and this leads to an issue called complete or quasi-complete separation problem in statistical modelling. This study aims to compare three methods suggested in the literature for the quasi-complete separation in a real small dataset; penalized maximum likelihood (Firth-type), exact logistic regression and bayesian logistic regression. Methods were compared via odds ratios, odds’ standard error estimates, confidence intervals and statistical significance. Parameter estimates were obtained under three different models with binary and continuous variables. Results show that all methods can provide convergence in the presence of quasi-complete separation. In conclusion, bayesian logistic regression estimates tend to be superior than the other methods in terms of estimation of standard errors.

Jackknife empirical likelihood confidence intervals for assessing heterogeneity in meta-analysis of rare binary event data

In meta-analysis, the heterogeneity of effect sizes across component studies is typically described by a variance parameter in a random-effects (Re) model. In the literature, methods for constructing confidence intervals (CIs) for the parameter often assume that study-level effect sizes are normally distributed. However, this assumption might be violated in practice, especially in meta-analysis of rare binary events. We propose to use jackknife empirical likelihood (JEL), a nonparametric approach that uses jackknife pseudo-values, to construct CIs for the heterogeneity parameter. To compute jackknife pseudo-values, we employ a moment-based estimator and consider two commonly used weighing schemes (i.e., equal and inverse variance weights). We prove that with each scheme, the resulting log empirical likelihood ratio follows a chi-square distribution asymptotically. We further examine the performance of the proposed JEL methods and compare them with existing CIs through simulation studies and data examples that focus on data of rare binary events. Our numerical results suggest that the JEL method with equal weights compares favorably to alternatives, especially when (observed) effect sizes are non-normal and the number of component studies is large. Thus, it is worth serious consideration in statistical inference.

Estimating group fixed effects in panel data with a binary dependent variable: How the LPM outperforms logistic regression in rare events data

Estimating fixed effects models can be challenging with rare events data. Researchers often face difficult trade-offs when selecting between the Linear Probability Model (LPM), logistic regression with group intercepts and the conditional logit. In this paper, I survey these tradeoffs and argue that, in fact, the LPM with fixed effects produces more accurate estimates and predicted probabilities than maximum likelihood specifications when the dependent variable has less than 25 percent of ones. I use Monte Carlo simulations to show when the LPM with fixed effects should be preferred. I perform these simulations on common time-series cross-sectional (TSCS) data structures found in the literature as well as big data. This paper provides clarity around fixed effects models in TSCS data and a novel technique to identify which one to use as a function of the frequency of events in y.

Psychosocial Determinants of Burn-Related Suicide: Evidence From the National Violent Death Reporting System.

Suicide is a unique phenomenon in humans. In 2017, over 47,000 Americans died by suicide, prompting a public health crisis. Suicide by burning, while one of the rarest forms of suicide, could be mitigated in part by early recognition and treatment of the underlying psychosocial factors. To date, investigators have attempted to tease out the risk factors associated with this phenomenon through the examination of burn center chart reviews and repository data, but these studies were limited by small sample sizes. In order to fill this gap in the literature, the authors analyzed 14 years of data (2003-2016) from the National Violent Death Reporting System. Suicides by burning (n = 722) were compared with a reference group containing all other suicide types (n = 166,949). Socio-demographic variables and psychosocial factors (eg, mental health, substance use, and alcohol use) were included in the model. Due to the imbalance between the target and reference groups, rare event data analysis was used to reduce potential small sample estimation bias. Results indicated that females (OR = 1.6, 95% CI: .50-2.83, P < .001), Caucasians (OR = 1.7, 95% CI: 1.1-2.7, P < .001), victims with schizophrenia (OR = 5.4, 95% CI: 3.7-7.8, P < .001), and victims with eating disorders (OR = 5.6, 95% CI: 1.9-8.6, P < .001) are significantly more likely to commit suicide by burning. These results contribute new knowledge to what is currently known about suicidal burn-related behaviors and supports interventions that focus on mitigating this insidious public health problem.

A Comparative Study of the Bias Correction Methods for Differential Item Functioning Analysis in Logistic Regression with Rare Events Data.

The logistic regression (LR) model for assessing differential item functioning (DIF) is highly dependent on the asymptotic sampling distributions. However, for rare events data, the maximum likelihood estimation method may be biased and the asymptotic distributions may not be reliable. In this study, the performance of the regular maximum likelihood (ML) estimation is compared with two bias correction methods including weighted logistic regression (WLR) and Firth's penalized maximum likelihood (PML) to assess DIF for imbalanced or rare events data. The power and type I error rate of the LR model for detecting DIF were investigated under different combinations of sample size, moderate and severe magnitudes of uniform DIF (DIF = 0.4 and 0.8), sample size ratio, number of items, and the imbalanced degree (τ). Indeed, as compared with WLR and for severe imbalanced degree (τ = 0.069), there were reductions of approximately 30% and 24% under DIF = 0.4 and 27% and 23% under DIF = 0.8 in the power of the PML and ML, respectively. The present study revealed that the WLR outperforms both the ML and PML estimation methods when logistic regression is used to evaluate DIF for imbalanced or rare events data.

Precluding rare outcomes by predicting their absence.

Forecasting extremely rare events is a pressing problem, but efforts to model such outcomes are often limited by the presence of multiple causes within classes of events, insufficient observations of the outcome to assess fit, and biased estimates due to insufficient observations of the outcome. We introduce a novel approach for analyzing rare event data that addresses these challenges by turning attention to the conditions under which rare outcomes do not occur. We detail how configurational methods can be used to identify conditions or sets of conditions that would preclude the occurrence of a rare outcome. Results from Monte Carlo experiments show that our approach can be used to systematically preclude up to 78.6% of observations, and application to ground-truth data coupled with a bootstrap inferential test illustrates how our approach can also yield novel substantive insights that are obscured by standard statistical analyses.

Logistic Regression Procedure Using Penalized Maximum Likelihood Estimation for Differential Item Functioning

AbstractIn the logistic regression (LR) procedure for differential item functioning (DIF), the parameters of LR have often been estimated using maximum likelihood (ML) estimation. However, ML estimation suffers from the finite‐sample bias. Furthermore, ML estimation for LR can be substantially biased in the presence of rare event data. The bias of ML estimation due to small samples and rare event data can degrade the performance of the LR procedure, especially when testing the DIF of difficult items in small samples. Penalized ML (PML) estimation was originally developed to reduce the finite‐sample bias of conventional ML estimation and also was known to reduce the bias in the estimation of LR for the rare events data. The goal of this study is to compare the performances of the LR procedures based on the ML and PML estimation in terms of the statistical power and Type I error. In a simulation study, Swaminathan and Rogers's Wald test based on PML estimation (PSR) showed the highest statistical power in most of the simulation conditions, and LRT based on conventional PML estimation (PLRT) showed the most robust and stable Type I error. The discussion about the trade‐off between bias and variance is presented in the discussion section.

Assessing correlates of protection in vaccine trials: statistical solutions in the context of high vaccine efficacy

BackgroundThe use of correlates of protection (CoPs) in vaccination trials offers significant advantages as useful clinical endpoint substitutes. Vaccines with very high vaccine efficacy (VE) are documented in the literature (VE ≥95%). The rare events (number of infections) observed in the vaccinated groups of these trials posed challenges when applying conventionally-used statistical methods for CoP assessment. In this paper, we describe the nature of these challenges, and propose easy-to-implement and uniquely-tailored statistical solutions for the assessment of CoPs in the specific context of high VE.MethodsThe Prentice criteria and meta-analytic frameworks are standard statistical methods for assessing vaccine CoPs, but can be problematic in high VE cases due to the rare events data available. As a result, lack of fit and the problem of infinite estimates may arise, in the former and latter methods respectively. The use of flexible models within the Prentice framework, and penalized-likelihood methods to solve the issue of infinite estimates can improve the performance of both methods in high VE settings.ResultsWe have 1) devised flexible non-linear models to counteract the Prentice framework lack of fit, providing sufficient statistical power to the method, and 2) proposed the use of penalised likelihood approaches to make the meta-analytic framework applicable on randomized subgroups, such as regions. The performance of the proposed methods for high VE cases was evaluated by running simulations.ConclusionsAs vaccines with high efficacy are documented in the literature, there is a need to identify effective statistical solutions to assess CoPs. Our proposed adaptations are straight-forward and improve the performance of conventional statistical methods for high VE data, leading to more reliable CoP assessments in the context of high VE settings.