Resampling Approach Research Articles

Predicting Multiple Outcomes Associated with Frailty based on Imbalanced Multi-label Classification.

Frailty syndrome is prevalent among the elderly, often linked to chronic diseases and resulting in various adverse health outcomes. Existing research has predominantly focused on predicting individual frailty-related outcomes. However, this paper takes a novel approach by framing frailty as a multi-label learning problem, aiming to predict multiple adverse outcomes simultaneously. In the context of multi-label classification, dealing with imbalanced label distribution poses inherent challenges to multi-label prediction. To address this issue, our study proposes a hybrid resampling approach tailored for handling imbalance problems in the multi-label scenario. The proposed resampling technique and prediction tasks were applied to a high-dimensional real-life medical dataset comprising individuals aged 65years and above. Several multi-label algorithms were employed in the experiment, and their performance was evaluated using multi-label metrics. The results obtained through our proposed approach revealed that the best-performing prediction model achieved an average precision score of 83%. These findings underscore the effectiveness of our method in predicting multiple frailty outcomes from a complex and imbalanced multi-label dataset.

Resampling approach for imbalanced data classification based on class instance density per feature value intervals

In practical applications, imbalanced datasets significantly degrade the classification performance of machine learning models. However, most conventional resampling approaches fall short in adequately addressing the varying contributions of individual features to the classification model. In response to this defect, this study introduces three novel resampling approaches. The first approach, Oversampling based on class instance density per feature value intervals (OCF), focuses on augmenting the dataset. The second approach, Undersampling based on class instance density per feature value intervals (UCF), seeks to reduce dataset size. The third approach, Hybrid sampling based on class instance density per feature value intervals (HSCF), which can perform oversampling and undersampling simultaneously. These approaches categorize feature value into different intervals based on their varying information content, calculate class instance densities within these intervals, and generate feature values in intervals with high discriminative information. Subsequently, these generated features are combined to synthesize minority class data, effectively achieving oversampling. Additionally, the study combines class instance density and feature importance to identify majority class data at the classification boundary with minimal contribution and subsequently executes undersampling. The flexibility to adjust sampling ratios and the integration of OCF and UCF enable the implementation of hybrid sampling. Finally, experiments on the benchmark dataset demonstrate the superiority and effectiveness of the proposed method. Furthermore, it is observed that the method proposed in this study enhances the feature dividing capability of decision tree classifiers. Hence, the best results are achieved when working in synergy with decision tree classifiers, leading to the most significant improvements in classification performance. All codes have been published at https://github.com/Wangfeiopen/HSCF.

A Novel Hybrid Resampling Approach to Address Class-Imbalanced Issues

A Novel Hybrid Resampling Approach to Address Class-Imbalanced Issues

Pre-natal and early life lead exposure and childhood inhibitory control: an item response theory approach to improve measurement precision of inhibitory control

BackgroundNeurodevelopmental performance tasks are often separately analyzed, even when they tap into a similar construct. This may yield mixed findings for associations of an exposure-neurobehavioral outcome. We develop an item response theory (IRT) approach to integrate multiple task variables together to improve measurement precision of the underlying construct. We apply this approach to create an integrative measure of childhood inhibitory control, and study impacts of pre/post-natal lead exposure.MethodsUsing data from a prospective cohort based in Mexico (N = 533), we created an inhibitory control scale that integrates accuracy and reaction time information from four inhibitory control tasks (Go/NoGo Letter, Go/NoGo Neutral, Go/NoGo Happy, Delis-Kaplan Executive Function System (D-KEFS) Color-Word Interference Test, Condition 3). Using a generalized partial credit item response theory model, we estimated an inhibitory control index for each participant. We then assessed adjusted associations between umbilical cord blood and 4-year lead and childhood inhibitory control. We developed a resampling approach to incorporate error estimates from the inhibitory control variable to confirm the consistency of the lead-inhibitory control associations. We modeled time-varying associations of lead with each inhibitory control measure separately.ResultsParticipants had a median age of 9 years; 51.4% were males. Umbilical cord blood [-0.06 (95% CI: -0.11, -0.01)] and 4-year lead [-0.07 (95% CI: -0.12, -0.02)] were associated with inhibitory control index at 8–10 years. A resampling approach confirmed that 4-year lead was consistently associated with childhood inhibitory control index. Umbilical cord blood and 4-year lead were each associated with 3 out of 8 measures in separate models.ConclusionThis is the first application of IRT in environmental epidemiology to create a latent variable for inhibitory control that integrates accuracy and reaction time information from multiple, related tasks. This framework can be applied to other correlated neurobehavioral assessments or other phenotype data.

Portfolio resampling in the Brazilian stock market

Markowitz optimization plays a crucial role in modern portfolio theory. However, it is well known that Markowitz optimization is highly affected by estimation errors in the mean vector and covariance matrix, resulting in extreme and/or unrealistic portfolio weights, lack of diversification, and poor out-of-sample performance. In response to these challenges, Michaud and Michaud (1998) proposed a heuristic portfolio resampling approach aimed to deliver more diversified and better out-of-sample portfolio performance. This approach has received both criticism and praise among academics and practitioners, with the main critique being the unclear economic benefits of applying this method to empirical data. To contribute to this ongoing debate, our study assess the performance of the Michaud and Michaud (1998) portfolio resampling approach using data from the Brazilian stock market. Additionally, we evaluate whether a resampling approach based on factor structure can yield superior out-of-sample performance. Our findings indicate no evidence of superiority over Markowitz optimization, reinforcing criticisms of the portfolio resampling approach.

CARBO: Clustering and rotation based oversampling for class imbalance learning

Class imbalance of a data set is a crucial problem in machine learning where one class significantly outnumbers others. In such a data set, classification is a troublesome task for the standard classification algorithms, leading to bias towards the majority class. Different methods have been developed so far, such as oversampling, undersampling, and cost-sensitive learning, to deal with class imbalance circumstances. Among these techniques, oversampling technique does not suffer from the information loss and critical cost selection challenges. However, appropriate synthetic sample generation can be challenging and vulnerable to privacy leakage. This research proposed an oversampling technique, called CARBO, using threshold-based geometric rotation and majority class influenced clustering. Unlike the existing resampling approaches to class imbalance problem, we contribute to consider the data privacy and optimal sample generation together for effective oversampling. The performance of CARBO is evaluated using 44 benchmark imbalanced data set. The empirical analysis elucidates that CARBO can make boosting-based C4.5 ensemble classifiers perform higher for 73% of the data set than six state-of-the-art approaches. In addition, the theoretical compatibility analysis of CARBO demonstrates its robustness.

Simultaneous Inference of Multiple Binary Endpoints in Biomedical Research: Small Sample Properties of Multiple Marginal Models and a Resampling Approach.

In biomedical research, the simultaneous inference of multiple binary endpoints may be of interest. In such cases, an appropriate multiplicity adjustment is required that controls the family-wise error rate, which represents the probability of making incorrect test decisions. In this paper, we investigate two approaches that perform single-step -value adjustments that also take into account the possible correlation between endpoints. A rather novel and flexible approach known as multiple marginal models is considered, which is based on stacking of the parameter estimates of the marginal models and deriving their joint asymptotic distribution. We also investigate a nonparametric vector-based resampling approach, and we compare both approaches with the Bonferroni method by examining the family-wise error rate and power for different parameter settings, including low proportions and small sample sizes. The results show that the resampling-based approach consistently outperforms the other methods in terms of power, while still controlling the family-wise error rate. The multiple marginal models approach, on the other hand, shows a more conservative behavior. However, it offers more versatility in application, allowing for more complex models or straightforward computation of simultaneous confidence intervals. The practical application of the methods is demonstrated using a toxicological dataset from the National ToxicologyProgram.

Handling Data Imbalance Problem in Hybrid Resampling Approach to Improve Accuracy of K-Nearest Neighbors Algorithm

Handling Data Imbalance Problem in Hybrid Resampling Approach to Improve Accuracy of K-Nearest Neighbors Algorithm

Optimal processing of surface facial EMG to identify emotional expressions: A data-driven approach

Surface facial electromyography (EMG) is commonly used to detect emotions from subtle facial expressions. Although there are established procedures for collecting EMG data and some aspects of their processing, there is little agreement among researchers about the optimal way to process the EMG signal, so that the study-unrelated variability (noise) is removed, and the emotion-related variability is best detected. The aim of the current paper was to establish an optimal processing pipeline for EMG data for identifying emotional expressions in facial muscles. We identified the most common processing steps from existing literature and created 72 processing pipelines that represented all the different processing choices. We applied these pipelines to a previously published dataset from a facial mimicry experiment, where 100 adult participants observed happy and sad facial expressions, whilst the activity of their facial muscles, zygomaticus major and corrugator supercilii, was recorded with EMG. We used a resampling approach and subsets of the original data to investigate the effect and robustness of different processing choices on the performance of a logistic regression model that predicted the mimicked emotion (happy/sad) from the EMG signal. In addition, we used a random forest model to identify the most important processing steps for the sensitivity of the logistic regression model. Three processing steps were found to be most impactful: baseline correction, standardisation within muscles, and standardisation within subjects. The chosen feature of interest and the signal averaging had little influence on the sensitivity to the effect. We recommend an optimal processing pipeline, share our code and data, and provide a step-by-step walkthrough for researchers.

Asymptotic properties of resampling‐based processes for the average treatment effect in observational studies with competing risks

AbstractIn observational studies with time‐to‐event outcomes, the g‐formula can be used to estimate a treatment effect in the presence of confounding factors. However, the asymptotic distribution of the corresponding stochastic process is complicated and thus not suitable for deriving confidence intervals or time‐simultaneous confidence bands for the average treatment effect. A common remedy are resampling‐based approximations, with Efron's nonparametric bootstrap being the standard tool in practice. We investigate the large sample properties of three different resampling approaches and prove their asymptotic validity in a setting with time‐to‐event data subject to competing risks. The usage of these approaches is demonstrated by an analysis of the effect of physical activity on the risk of knee replacement among patients with advanced knee osteoarthritis.

Multiplier subsample bootstrap for statistics of time series

Block-based bootstrap, block-based subsampling and multiplier bootstrap are three common nonparametric tools for statistical inference under dependent observations. Combining the ideas of those three, a novel resampling approach, the multiplier subsample bootstrap (MSB), is proposed. Instead of generating a resample from the observations, the MSB imitates the statistic by weighting the block-based subsample statistics with independent standard Gaussian random variables. Given the asymptotic normality of the statistic, the bootstrap validity is established under some mild moment conditions. Involving the idea of MSB, the other resampling approach, the hybrid multiplier subsampling periodogram bootstrap (HMP), is developed for mimicking frequency-domain spectral mean statistics in the paper. A simulation study demonstrates that both the MSB and HMP achieve good performance.

Statistical reanalysis of Archean zircon paleointensities: No evidence for stagnant-lid tectonics

The initiation of mobile-lid plate tectonics on Earth represented a critical transition towards a more familiar world in terms of surface temperature stabilization, biogeochemical cycling, topography creation, and other processes. Zircon-based estimates of the geomagnetic field intensity have recently been cited as providing evidence for the lack of mobile-lid motion between 3.9 and 3.4 billion years ago (Ga). We reanalyze the published dataset of 91 zircon paleointensities from the Jack Hills (Australia) and Green Sandstone Bed (GSB; South Africa) localities within this time interval and, using both analytical and bootstrap resampling approaches, show that the small number of samples result in large uncertainties in implied paleolatitude. Specifically, in more likely scenarios that do not assume coherent motion for both localities, all latitudinal displacements on Earth are permitted within the 95 % confidence interval. We also examine the less likely scenario that the two landmasses shared a motion history, which increases the data density and presents the best-case scenario for constraining latitudinal motion. In this case, the 95 % confidence interval of the zircon paleointensity data is compatible with the displacements of between 35 % and 52 % of modern continental localities, all of which experience mobile-lid tectonics. Finally, generating expected paleointensity time series for modern continents undergoing mobile-lid motion shows that about two-thirds of these motions would not be resolved by zircon paleointensities, even in the best-case scenario of combining Jack Hills and GSB datasets. All of these analyses assume that these zircons retain a primary paleomagnetic signal, an assertion which is opposed by a number of published zircon magnetism studies. We conclude that Archean zircon paleointensities do not provide evidence for or against mobile-lid plate tectonics prior to 3.4 Ga. Future paleomagnetic investigation of tectonic regime on the early Earth should therefore focus on magnetization directions in well-preserved, oriented whole rocks.

Resampling-based confidence intervals and bands for the average treatment effect in observational studies with competing risks

The g-formula can be used to estimate the treatment effect while accounting for confounding bias in observational studies. With regard to time-to-event endpoints, possibly subject to competing risks, the construction of valid pointwise confidence intervals and time-simultaneous confidence bands for the causal risk difference is complicated, however. A convenient solution is to approximate the asymptotic distribution of the corresponding stochastic process by means of resampling approaches. In this paper, we consider three different resampling methods, namely the classical nonparametric bootstrap, the influence function equipped with a resampling approach as well as a martingale-based bootstrap version, the so-called wild bootstrap. For the latter, three sub-versions based on differing distributions of the underlying random multipliers are examined. We set up a simulation study to compare the accuracy of the different techniques, which reveals that the wild bootstrap should in general be preferred if the sample size is moderate and sufficient data on the event of interest have been accrued. For illustration, the resampling methods are further applied to data on the long-term survival in patients with early-stage Hodgkin’s disease.

Leveraging Sampling Schemes on Skewed Class Distribution to Enhance Male Fertility Detection with Ensemble AI Learners

Designing effective AI models becomes a challenge when dealing with imbalanced/skewed class distributions in datasets. Addressing this, re-sampling techniques often come into play as potential solutions. In this investigation, we delve into the male fertility dataset, exploring 14 re-sampling approaches to understand their impact on enhancing predictive model performance. The research employs conventional AI learners to gauge male fertility potential. Notably, five ensemble AI learners are studied, their performances are compared, and their results are evaluated using four measurement indices. Through comprehensive comparative analysis, we identify substantial enhancement in model effectiveness. Our findings showcase that the LightGBM model with SMOTE-ENN re-sampling stands out, achieving an efficacy of 96.66% and an F1-Score of 95.60% through 5-fold cross-validation. Interestingly, the CatBoost model, without re-sampling, exhibits strong performance, achieving an efficacy of 86.99% and an F1-Score of 93.02%. Furthermore, we benchmark our approach against state-of-the-art methods in male fertility prediction, particularly highlighting the use of re-sampling techniques like SMOTE and ESLSMOTE. Consequently, our proposed model emerges as a robust and efficient computational framework, promising accurate male fertility prediction.

Examining the Relationship Between Multiple Tests of Receptive Vocabulary.

Numerous tasks have been developed to measure receptive vocabulary, many of which were designed to be administered in person with a trained researcher or clinician. The purpose of the current study is to compare a common, in-person test of vocabulary with other vocabulary assessments that can be self-administered. Fifty-three participants completed the Peabody Picture Vocabulary Test (PPVT) via online video call to mimic in-person administration, as well as four additional fully automated, self-administered measures of receptive vocabulary. Participants also completed three control tasks that do not measure receptive vocabulary. Pearson correlations indicated moderate correlations among most of the receptive vocabulary measures (approximately r = .50-.70). As expected, the control tasks revealed only weak correlations to the vocabulary measures. However, subsets of items of the four self-administered measures of receptive vocabulary achieved high correlations with the PPVT (r > .80). These subsets were found through a repeated resampling approach. Measures of receptive vocabulary differ in which items are included and in the assessment task (e.g., lexical decision, picture matching, synonym matching). The results of the current study suggest that several self-administered tasks are able to achieve high correlations with the PPVT when a subset of items are scored, rather than the full set of items. These data provide evidence that subsets of items on one behavioral assessment can more highly correlate to another measure. In practical terms, these data demonstrate that self-administered, automated measures of receptive vocabulary can be used as reasonable substitutes of at least one test (PPVT) that requires human interaction. That several of the fully automated measures resulted in high correlations with the PPVT suggests that different tasks could be selected depending on the needs of the researcher. It is important to note the aim was not to establish clinical relevance of these measures, but establish whether researchers could use an experimental task of receptive vocabulary that probes a similar construct to what is captured by the PPVT, and use these measures of individual differences.

Assessment of resampling methods on performance of landslide susceptibility predictions using machine learning in Kendari City, Indonesia

Abstract Landslide susceptibility projections that rely on independent models produce biased results. This situation will worsen class balance if working with a small population. This study proposes a landslide susceptibility prediction model based on resampling, cross-validation, bootstrap, and random subsampling approaches, which is integrated with the machine learning model, generalized linear model, support vector machine, random forest, boosted regression trees, classification and regression tree, multivariate adaptive regression splines, mixture discriminate analysis, flexible discriminant analysis, maximum entropy, and maximum likelihood. This methodology was applied in Kendari City, an urban area which faced destructive erosion. Area under the ROC curve (AUC), true skill statistics (TSS), correlation coefficient (COR), normalized mutual information (NMI), and correct classification rate (CCR) were used to evaluate the predictive accuracy of the proposed model. The results show that the resampling algorithm improves the performance of the standalone model. Results also revealed that standalone models had better performance with the CV algorithm compared to the Bt and RS algorithms. The Bt-RF model excels in statistical measures (AUC = 0.97, TSS = 0.97, COR = 0.99, NMI = 0.50, and CCR = 0.93). Given the admirable performance of the proposed models in a moderate scale area, promising results can be expected from these models for other regions.

Demand Side Perception on Success Factors for Implementing Public Road Construction Projects

Cost and schedule overruns, as well as substandard works, are common factors that fail organisations’ objectives within the construction industry. Particularly, these factors affect public road construction projects and cost taxpayers. Researchers continue scanning the environment to establish why construction projects are ever behind schedule and over budget with substandard works and contract variations to identify significant factors for successful project implementation. This study expands the debate by looking at demand side perception in establishing success factors for implementing public road construction projects motivated by governments’ high expenditure on construction projects without meeting objectives. A cross-sectional research design with a structured self-administered questionnaire was used to obtain views from three public entities representing the demand side. Results were analysed following partial least square structural modelling (PLS-SEM) in Smart-PLS3. The research design enabled statistical tests to be conducted on validity, reliability, normality, multicollinearity, correlations and regression. The PLS algorithm and bootstrapping resampling approach were employed to determine the relationship between variables by estimating path coefficients and significance. Path coefficients helped to determine strength, direction and significance, and examine the variance of dependent variables explained by combined independent variables. Results revealed that the professionalism of the staff, compliance with the public procurement regulatory framework, monitoring activities and contractors’ resistance to non-compliance are significant success factors in enhancing public road implementation. Hence, adopting these factors would be a game changer in implementing complex road construction projects. In addition, complex construction projects in a dynamic construction industry require continuous scanning to establish more factors and cope with industry dynamics.

Optimizing Neural Networks for Academic Performance Classification Using Feature Selection and Resampling Approach

The features present in large datasets significantly affect the performance of machine learning models. Redundant and irrelevant features will be rejected and cause a decrease in machine learning model performance. This paper proposes HyFeS-ROS-ANN: Hybrid Feature Selection and Resampling combination method for binary classification using artificial neural network multilayer perceptron (MLP). The first stage of this approach is to use a combination of two feature selection methods to select essential features that are highly correlated with model performance. The second stage of this approach is to use a combination of resampling methods to handle unbalanced data classes. Both approaches are applied to the academic performance classification model using the MLP neural network. This research dataset is obtained using three-dimensional (3D) frameworks such as the Big Five Personality to determine the Personality that affects academic performance from the student dimension, the Family Influence Scale (FIS), which measures factors that affect academic performance from the family dimension, and Higher Education Institutions Service Quality (HEISQUAL) to measure service quality and its influence on academic performance from the Education institution dimension. Previous research shows that the CoR-ANN algorithm has a model accuracy rate of 94%. The research results based on the dataset show that our proposed method can improve accuracy by selecting more relevant and essential features in improving model performance. The results show that the features are reduced from 135 to 108, while the HyFS-ROS-ANN model for binary classification accuracy increases to 100%.

Unbiased Proxy Calibration

The linear calibration model is a powerful statistical tool that can be utilized to predict an unknown response variable, Y, through observations of a proxy or predictor variable, X. Since calibration involves estimation of regression model parameters on the basis of a limited amount of noisy data, an unbiased calibration slope estimation is of utmost importance. This can be achieved by means of state-of-the-art, data-driven statistical techniques. The present paper shows that weighted least-squares for both variables estimation (WLSXY) is able to deliver unbiased slope estimations under heteroscedasticity. In the case of homoscedasticity, besides WLSXY, ordinary least-squares (OLS) estimation with bias correction (OLSBC) also performs well. For achieving unbiasedness, it is further necessary to take the correct regression direction (i.e., of Y on X) into account. The present paper introduces a pairwise moving block bootstrap resampling approach for obtaining accurate estimation confidence intervals (CIs) under real-world climate conditions (i.e., non-Gaussian distributional shapes and autocorrelations in the noise components). A Monte Carlo simulation experiment confirms the feasibility and validity of this approach. The parameter estimates and bootstrap replications serve to predict the response with CIs. The methodological approach to unbiased calibration is illustrated for a paired time series dataset of sea-surface temperature and coral oxygen isotopic composition. Fortran software with implementation of OLSBC and WLSXY accompanies this paper.

Practical Considerations for Sandwich Variance Estimation in 2-Stage Regression Settings.

In this paper, we present a practical approach for computing the sandwich variance estimator in 2-stage regression model settings. As a motivating example for 2-stage regression, we consider regression calibration, a popular approach for addressing covariate measurement error. The sandwich variance approach has rarely been applied in regression calibration, despite its requiring less computation time than popular resampling approaches for variance estimation, specifically the bootstrap. This is probably because it requires specialized statistical coding. Here we first outline the steps needed to compute the sandwich variance estimator. We then develop a convenient method of computation in R for sandwich variance estimation, which leverages standard regression model outputs and existing R functions and can be applied in the case of a simple random sample or complex survey design. We use a simulation study to compare the sandwich estimator to a resampling variance approach for both settings. Finally, we further compare these 2 variance estimation approaches in data examples from the Women's Health Initiative (1993-2005) and the Hispanic Community Health Study/Study of Latinos (2008-2011). In our simulations, the sandwich variance estimator typically had good numerical performance, but simple Wald bootstrap confidence intervals were unstable or overcovered in certain settings, particularly when there was high correlation between covariates or large measurement error.