Resampling Method Research Articles

SPE-ACGAN: A Resampling Approach for Class Imbalance Problem in Network Intrusion Detection Systems

Network Intrusion Detection Systems (NIDSs) play a vital role in detecting and stopping network attacks. However, the prevalent imbalance of training samples in network traffic interferes with NIDS detection performance. This paper proposes a resampling method based on Self-Paced Ensemble and Auxiliary Classifier Generative Adversarial Networks (SPE-ACGAN) to address the imbalance problem of sample classes. To deal with the class imbalance problem, SPE-ACGAN oversamples the minority class samples by ACGAN and undersamples the majority class samples by SPE. In addition, we merged the CICIDS-2017 dataset and the CICIDS-2018 dataset into a more imbalanced dataset named CICIDS-17-18 and validated the effectiveness of the proposed method using the three datasets mentioned above. SPE-ACGAN is more effective than other resampling methods in improving NIDS detection performance. In particular, SPE-ACGAN improved the F1-score of Random Forest, CNN, GoogLeNet, and CNN + WDLSTM by 5.59%, 3.75%, 3.60%, and 3.56% after resampling.

Bankruptcy prediction model using cost-sensitive extreme gradient boosting in the context of imbalanced datasets

<span lang="EN-US">In the process of bankruptcy prediction models, a class imbalanced problem has occurred which limits the performance of the models. Most prior research addressed the problem by applying resampling methods such as the synthetic minority oversampling technique (SMOTE). However, resampling methods lead to other issues, e.g., increasing noisy data and training time during the process. To improve the bankruptcy prediction model, we propose cost-sensitive extreme gradient boosting (CS-XGB) to address the class imbalanced problem without requiring any resampling method. The proposed method’s effectiveness is evaluated on six real-world datasets, i.e., the LendingClub, and five Polish companies’ bankruptcy. This research compares the performance of CS-XGB with other ensemble methods, including SMOTE-XGB which applies SMOTE to the training set before the learning process. The experimental results show that i) based on LendingClub, the CS-XGB improves the performance of XGBoost and SMOTE-XGB by more than 50% and 33% on bankruptcy detection rate (BDR) and geometric mean (GM), respectively, and ii) the CS-XGB model outperforms random forest (RF), Bagging, AdaBoost, XGBoost, and SMOTE-XGB in terms of BDR, GM, and the area under a receiver operating characteristic curve (AUC) based on the five Polish datasets. Besides, the CS-XGB model achieves good overall prediction results.</span>

Assessing North Atlantic Tropical Cyclone Rainfall Hazard Using Engineered-Synthetic Storms and a Physics-Based Tropical Cyclone Rainfall Model

Abstract In this study, we design a statistical method to couple observations with a physics-based tropical cyclone (TC) rainfall model (TCR) and engineered-synthetic storms for assessing TC rainfall hazard. We first propose a bias-correction method to minimize the errors induced by TCR via matching the probability distribution of TCR-simulated historical TC rainfall with gauge observations. Then we assign occurrence probabilities to engineered-synthetic storms to reflect local climatology, through a resampling method that matches the probability distribution of a newly proposed storm parameter named rainfall potential (POT) in the synthetic dataset with that in the observation. POT is constructed to include several important storm parameters for TC rainfall such as TC intensity, duration, and distance and environmental humidity near landfall, and it is shown to be correlated with TCR-simulated rainfall. The proposed method has a satisfactory performance in reproducing the rainfall hazard curve in various locations in the continental United States; it is an improvement over the traditional joint probability method (JPM) for TC rainfall hazard assessment.

Diagnosis of the Diseases Using Resampling Methods with Machine Learning Algorithms

The rapid diagnosis of diseases is very important for the early start of the treatment process. Pneumonia is a disease that affects the lungs and can cause death in advanced cases. Pneumonia is still today diagnosed by doctors while examining chest X-ray images. As diagnosis of the diseases using machine learning algorithms will be useful. In addition, high success of rate will be obtained while classifying balanced dataset by using machine learning algorithms. Resampling methods are used to balance the dataset by using under-sampling or over-sampling methods. In literature, there is no study comparing under-sampling and over-sampling methods. In the study, an open source dataset was used which included two classes published through the Kaggle data store. The data set includes 1341 healthy and 3875 pneumonia chest X-ray images. Two different resampling methods named Random under-sampling (RU) and ADASYN (Adaptive Synthetic) over-sampling were used while balancing healthy and pneumonia images. After this operation obtained data were used for training machine learning algorithms. In the study, first and second level attributes of the X-ray images in the dataset were used. Logistic Regression (LR) and Support Vector Machine (SVM) algorithms were used for classification of dataset. According to the results obtained, 93.109% accuracy rate of classification success was achieved of X-ray image dataset which balanced by ADASYN over-sampling method with SVM algorithms.

Joint modelling of quantile regression for longitudinal data with information observation times and a terminal event

AbstractLongitudinal data arise frequently in biomedical follow‐up observation studies. Conditional mean regression and conditional quantile regression are two popular approaches to model longitudinal data. Many results are derived under the case where the response variables are independent of the observation times. In this article, we propose a quantile regression model for the analysis of longitudinal data, where the longitudinal responses are allowed to not only depend on the past observation history but also associate with a terminal event (e.g., death). Non‐smoothing estimating equation approaches are developed to estimate parameters, and the consistency and asymptotic normality of the proposed estimators are established. The asymptotic variance is estimated by a resampling method. A majorize‐minimize algorithm is proposed to compute the proposed estimators. Simulation studies show that the proposed estimators perform well, and an HIV‐RNA dataset is used to illustrate the proposed method.

Development and validation of a prediction model based on machine learning algorithms for predicting the risk of heart failure in middle-aged and older US people with prediabetes or diabetes.

The purpose of this study was to develop and validate a machine learning (ML) based prediction model for the risk of heart failure (HF) in patients with prediabetes or diabetes. We used 3527 subjects aged 40 years and older with a prior diagnosis of prediabetes or diabetes from the National Health and Nutrition Examination Survey (NHANES) from 2007 to 2018. The search for independent risk variables linked to HF was conducted using univariate and multivariable logistic regression analysis. The 3527 subjects were randomly divided into training set and validation set in a 7:3 ratio. Five ML models were built on the training set using five ML algorithms, including random forest (RF), and then validated on the validation set. Receiver operating characteristic (ROC) curves, calibration curves, and decision curve analysis and Bootstrap resampling method were used to measure the predictive performance of the five ML models. Multivariate logistic regression analysis showed that age, poverty-to-income ratio, myocardial infarction condition, coronary heart disease condition, chest pain condition, and glucose-lowering medication use were independent predictors of HF. By comparing the performance of the five ML models, the RF model (AUC = 0.978) was the best prediction model. The risk of HF in middle-aged and elderly patients with prediabetes or diabetes can be accurately predicted using ML models. The best prediction performance is presented by RF model, which can assist doctors in making clinical decisions.

Klasifikasi Penyakit Daun Jagung Menggunakan Lightweight Convolutional Neural Network

Deep Neural Networks have achieved great success in many tasks and research especially in pattern recognition. However, high computing and large memory requirements prevent them from being applied to limited devices and require large costs. And Imbalance of data in Machine Learning refers to the unequal distribution of classes in a data set. This problem is often encountered in classification tasks where the distribution of classes within a given data set is not uniform. Given these problems, the researchers proposed the Light Neural Network method and added the Resampling method to Corn Leaf Disease data, to ease the computational process and balance the data. Researchers use several methods in machine learning which involve the use of knowledge that has been learned by existing models (Transfer Learning), including: MobileNet, EfficientNetB1 and Exception, as well as conducting comparisons between models. These models use the Depthwise Separable Convolution technique to reduce the number of parameters and convolution operations that need to be performed, thereby reducing the computational requirements and model complexity. The results of light neural network research and the combination of Resampling, the highest Transfer Learning model of 95% obtained from the EfficientNetB1 model than other Transfer learning models.

DynaQ: online learning from imbalanced multi-class streams through dynamic sampling

AbstractOnline supervised learning from fast-evolving data streams, particularly in domains such as health, the environment, and manufacturing, is a crucial research area. However, these domains often experience class imbalance, which can skew class distributions. It is essential for online learning algorithms to analyze large datasets in real-time while accurately modeling rare or infrequent classes that may appear in bursts. While methods have been proposed to handle binary class imbalance, there is a lack of attention to multi-class imbalanced settings with varying degrees of imbalance in evolving streams. In this paper, we present the Dynamic Queues (DynaQ) algorithm for online learning in multi-class imbalanced settings to fill this knowledge gap. Our approach utilizes a batch-based resampling method that creates an instance queue for each class to balance the number of instances. We maintain a queue threshold and remove older samples during training. Additionally, we dynamically oversample minority classes based on one of four rate parameters: recall, F1-score, $$\kappa _m$$ κ m , and Euclidean distance. Our learning algorithm consists of an ensemble that uses sliding windows and a soft voting schema while incorporating a drift detection mechanism. Our experimental results demonstrate the superiority of the DynaQ approach over state-of-the-art methods.

Optimal selection of resampling methods for imbalanced data with high complexity.

Class imbalance is a major problem in classification, wherein the decision boundary is easily biased toward the majority class. A data-level solution (resampling) is one possible solution to this problem. However, several studies have shown that resampling methods can deteriorate the classification performance. This is because of the overgeneralization problem, which occurs when samples produced by the oversampling technique that should be represented in the minority class domain are introduced into the majority-class domain. This study shows that the overgeneralization problem is aggravated in complex data settings and introduces two alternate approaches to mitigate it. The first approach involves incorporating a filtering method into oversampling. The second approach is to apply undersampling. The main objective of this study is to provide guidance on selecting optimal resampling methods in imbalanced and complex datasets to improve classification performance. Simulation studies and real data analyses were performed to compare the resampling results in various scenarios with different complexities, imbalances, and sample sizes. In the case of noncomplex datasets, undersampling was found to be optimal. However, in the case of complex datasets, applying a filtering method to delete misallocated examples was optimal. In conclusion, this study can aid researchers in selecting the optimal method for resampling complex datasets.

Validation of machine learning ridge regression models using Monte Carlo, bootstrap, and variations in cross-validation

Abstract In recent years, there have been several calls by practitioners of machine learning to provide more guidelines on how to use its methods and techniques. For example, the current literature on resampling methods is confusing and sometimes contradictory; worse, there are sometimes no practical guidelines offered at all. To address this shortcoming, a simulation study was conducted that evaluated ridge regression models fitted on five real-world datasets. The study compared the performance of four resampling methods, namely, Monte Carlo resampling, bootstrap, k-fold cross-validation, and repeated k-fold cross-validation. The goal was to find the best-fitting λ (regularization) parameter that would minimize mean squared error, by using nine variations of these resampling methods. For each of the nine resampling variations, 1,000 runs were performed to see how often a good fit, average fit, and poor fit λ value would be chosen. The resampling method that chose good fit values the greatest number of times was deemed the best method. Based on the results of the investigation, three general recommendations are made: (1) repeated k-fold cross-validation is the best method to select as a general-purpose resampling method; (2) k = 10 folds is a good choice in k-fold cross-validation; (3) Monte Carlo and bootstrap are underperformers, so they are not recommended as general-purpose resampling methods. At the same time, no resampling method was found to be uniformly better than the others.

Comparison of Bootstrap Methods for Estimating Causality in Linear Dynamic Systems: A Review.

In this study, we present a thorough comparison of the performance of four different bootstrap methods for assessing the significance of causal analysis in time series data. For this purpose, multivariate simulated data are generated by a linear feedback system. The methods investigated are uncorrelated Phase Randomization Bootstrap (uPRB), which generates surrogate data with no cross-correlation between variables by randomizing the phase in the frequency domain; Time Shift Bootstrap (TSB), which generates surrogate data by randomizing the phase in the time domain; Stationary Bootstrap (SB), which calculates standard errors and constructs confidence regions for weakly dependent stationary observations; and AR-Sieve Bootstrap (ARSB), a resampling method based on AutoRegressive (AR) models that approximates the underlying data-generating process. The uPRB method accurately identifies variable interactions but fails to detect self-feedback in some variables. The TSB method, despite performing worse than uPRB, is unable to detect feedback between certain variables. The SB method gives consistent causality results, although its ability to detect self-feedback decreases, as the mean block width increases. The ARSB method shows superior performance, accurately detecting both self-feedback and causality across all variables. Regarding the analysis of the Impulse Response Function (IRF), only the ARSB method succeeds in detecting both self-feedback and causality in all variables, aligning well with the connectivity diagram. Other methods, however, show considerable variations in detection performance, with some detecting false positives and others only detecting self-feedback.

Risk factors for rebleeding in gastroduodenal ulcers.

Rebleeding after hemostasis of the gastroduodenal ulcer (GDU) is one of the indicators associated with death among GDU patients. However, there are few studies on risk score that contribute to rebleeding after endoscopic hemostasis of bleeding peptic ulcers. The aim of this study was to identify factors associated with rebleeding, including patient factors, after endoscopic hemostasis of bleeding gastroduodenal ulcers and to stratify the risk of rebleeding. We retrospectively enrolled 587 consecutive patients who were treated for Forrest Ia to IIa bleeding gastroduodenal ulcers with endoscopic hemostasis at three institutions. Risk factors associated with rebleeding were assessed using univariate and multivariate logistic regression analyses. The Rebleeding Nagoya University (Rebleeding-N) scoring system was developed based on the extracted factors. The Rebleeding-N score was internally validated using bootstrap resampling methods. Sixty-four patients (11%) had rebleeding after hemostasis of gastroduodenal ulcers. Multivariate logistic regression analysis revealed four independent rebleeding risk factors: blood transfusion, albumin <2.5, duodenal ulcer, and diameter of the exposed vessel ≧2mm. Patients with 4 risk factors in the Rebleeding-N score had a 54% rebleeding rate, and patients with 3 risk factors had 44% and 25% rebleeding rates. In the internal validation, the mean area under the curve of the Rebleeding-N score was 0.830 (95% CI = 0.786-0.870). Rebleeding after clip hemostasis of bleeding gastroduodenal ulcers was associated with blood transfusion, albumin <2.5, diameter of the exposed vessel ≧2mm, and duodenal ulcer. The Rebleeding-N score was able to stratify the risk of rebleeding.

Homography Matrix-Based Local Motion Consistent Matching for Remote Sensing Images

Feature matching is a fundamental task in the field of image processing, aimed at ensuring correct correspondence between two sets of features. Putative matches constructed based on the similarity of descriptors always contain a large number of false matches. To eliminate these false matches, we propose a remote sensing image feature matching method called LMC (local motion consistency), where local motion consistency refers to the property that adjacent correct matches have the same motion. The core idea of LMC is to find neighborhoods with correct motion trends and retain matches with the same motion. To achieve this, we design a local geometric constraint using a homography matrix to represent local motion consistency. This constraint has projective invariance and is applicable to various types of transformations. To avoid outliers affecting the search for neighborhoods with correct motion, we introduce a resampling method to construct neighborhoods. Moreover, we design a jump-out mechanism to exit the loop without searching all possible cases, thereby reducing runtime. LMC can process over 1000 putative matches within 100 ms. Experimental evaluations on diverse image datasets, including SUIRD, RS, and DTU, demonstrate that LMC achieves a higher F-score and superior overall matching performance compared to state-of-the-art methods.

Best Practices in Supervised Machine Learning: A Tutorial for Psychologists

Supervised machine learning (ML) is becoming an influential analytical method in psychology and other social sciences. However, theoretical ML concepts and predictive-modeling techniques are not yet widely taught in psychology programs. This tutorial is intended to provide an intuitive but thorough primer and introduction to supervised ML for psychologists in four consecutive modules. After introducing the basic terminology and mindset of supervised ML, in Module 1, we cover how to use resampling methods to evaluate the performance of ML models (bias-variance trade-off, performance measures, k-fold cross-validation). In Module 2, we introduce the nonlinear random forest, a type of ML model that is particularly user-friendly and well suited to predicting psychological outcomes. Module 3 is about performing empirical benchmark experiments (comparing the performance of several ML models on multiple data sets). Finally, in Module 4, we discuss the interpretation of ML models, including permutation variable importance measures, effect plots (partial-dependence plots, individual conditional-expectation profiles), and the concept of model fairness. Throughout the tutorial, intuitive descriptions of theoretical concepts are provided, with as few mathematical formulas as possible, and followed by code examples using the mlr3 and companion packages in R. Key practical-analysis steps are demonstrated on the publicly available PhoneStudy data set ( N = 624), which includes more than 1,800 variables from smartphone sensing to predict Big Five personality trait scores. The article contains a checklist to be used as a reminder of important elements when performing, reporting, or reviewing ML analyses in psychology. Additional examples and more advanced concepts are demonstrated in online materials ( https://osf.io/9273g/ ).

A fast re-sampling method for using reliability ratings of sightings with extinction-date estimators: Comment.

A fast re-sampling method for using reliability ratings of sightings with extinction-date estimators: Comment.

Fast Resampling and Monte Carlo Methods in Python

Fast Resampling and Monte Carlo Methods in Python

Censored Interquantile Regression Model with Time-Dependent Covariates

Conventionally, censored quantile regression stipulates a specific, pointwise conditional quantile of the survival time given covariates. Despite its model flexibility and straightforward interpretation, the pointwise formulation oftentimes yields rather unstable estimates across neighboring quantile levels with large variances. In view of this phenomenon, we propose a new class of quantile-based regression models with time-dependent covariates for censored data. The models proposed aim to capture the relationship between the failure time and the covariate processes of a target population that falls within a specific quantile bracket. The pooling of information within a homogeneous neighborhood facilitates more efficient estimates hence, more consistent conclusion on statistical significances of the variables concerned. This new formulation can also be regarded as a generalization of the accelerated failure time model for survival data in the sense that it relaxes the assumption of global homogeneity for the error at all quantile levels. By introducing a class of weighted rank-based estimation procedure, our framework allows a quantile-based inference on the covariate effect with a less restrictive set of assumptions. Numerical studies demonstrate that the proposed estimator outperforms existing alternatives under various settings in terms of smaller empirical biases and standard deviations. A perturbation-based resampling method is also developed to reconcile the asymptotic distribution of the parameter estimates. Finally, consistency and weak convergence of the proposed estimator are established via empirical process theory. Supplementary materials for this article are available online.

Spatial or Random Cross-Validation? The Effect of Resampling Methods in Predicting Groundwater Salinity with Machine Learning in Mediterranean Region

Machine learning (ML) algorithms are extensively used with outstanding prediction accuracy. However, in some cases, their overfitting capabilities, along with inadvertent biases, might produce overly optimistic results. Spatial data are a special kind of data that could introduce biases to ML due to their intrinsic spatial autocorrelation. To address this issue, a special resampling method has emerged called spatial cross-validation (SCV). The purpose of this study was to evaluate the performance of SCV compared with conventional random cross-validation (CCV) used in most ML studies. Multiple ML models were created with CCV and SCV to predict groundwater electrical conductivity (EC) with data (A) from Rhodope, Greece, in the summer of 2020; (B) from the same area but at a different time (summer 2019); and (C) from a new area (the Salento peninsula, Italy). The results showed that the SCV provides ML models with superior generalization capabilities and, hence, better prediction results in new unknown data. The SCV seems to be able to capture the spatial patterns in the data while also reducing the over-optimism bias that is often associated with CCV methods. Based on the results, SCV could be applied with ML in studies that use spatial data.

A Weight-Based Cutoff Resampling Method for Accelerated Particle Filtering

The particle filter is known to be a powerful tool for the estimation of moving targets guided by nonlinear dynamics and sensors. The filter, however, is known to suffer from degeneracy — a feature of one particle gathering all the weight, thus causing the filter to completely diverge. Degeneracy problems become more evident when the sensors are accurate and/or the target maneuvers greatly. The resampling step in the particle filter is critical because it avoids degeneracy of particles by eliminating the wasteful use of particles that do not contribute to the posterior probability density function. The conventional resampling methods, despite being unbiased in approximating the posterior density, involve exhaustive and sequential communication within the particles and thus are computationally expensive. Hence conventional resampling is a major bottleneck for fast implementation of particle filters for real-time tracking. In this paper, we propose a new approach of filtering that requires resampling of only a minimum number of the most important particles that contribute to the posterior density. Minimizing the resampling operation to over a few important particles substantially accelerates the filtering process. We show the merits of the proposed method via simulations using a nonlinear example.

Extended natural neighborhood for SMOTE and its variants in imbalanced classification

Imbalanced data classification is a challenging issue encountered in many practical applications. Synthetic minority oversampling technique (SMOTE) and its variants are popular resampling methods. However, in most of these methods, the neighborhood determined by k-nearest neighbor (kNN) cannot reflect the local distribution precisely, leading to the generation of noisy examples. To solve this problem, we propose a neighborhood concept without parameter k called extended natural neighbor (ENaN), which is derived from natural neighbor (NaN). ENaN unites kNN and reverse kNN to determine neighbors adaptively according to the sample distribution. Compared to NaN, ENaN explores broad neighborhoods, which facilitates to improve the quality of generated examples. ENaN-based SMOTE (ENaNSMOTE) can improve the sample distribution obtained by SMOTE and NaNSMOTE. Extensive experiments using 30 synthetic and 20 real-world datasets prove the effectiveness of ENaN in SMOTE and its variants.