Oversampling Strategy Research Articles

A novel self-training framework for semi-supervised soft sensor modeling based on indeterminate variational autoencoder

In modern industrial processes, the high acquisition cost of labeled data can lead to a large number of unlabeled samples, which greatly impacts the accuracy of traditional soft sensor models. To this end, this paper proposes a novel semi-supervised soft sensor framework that can fully utilize the unlabeled data to expand the original labeled data, and ultimately improve the prediction accuracy. Specifically, an indeterminate variational autoencoder (IVAE) is first proposed to obtain the pseudo-labels of unlabeled data as well as their uncertainty. On this basis, the IVAE-based self-training (ST-IVAE) framework is further naturally proposed to expand the original small labeled dataset through continuous circulation. Among them, a variance-based oversampling (VOS) strategy is introduced to better utilize the pseudo-label uncertainty. By determining similar sample sets through the comparison of Kullback-Leibler (KL) divergence obtained by the proposed IVAE model, each sample can be independently modeled for prediction. The effectiveness of the proposed semi-supervised framework is verified on two real industrial processes and the realization results illustrate that the ST-IVAE framework can still predict well even in the presence of missing input data compared to state-of-the-art methodologies in addressing semi-supervised soft sensing challenges.

Deep Representation Learning With Sample Generation and Augmented Attention Module for Imbalanced ECG Classification.

Developing an efficient heartbeat monitoring system has become a focal point in numerous healthcare applications. Specifically, in the last few years, heartbeat classification for arrhythmia detection has gained considerable interest from researchers. This paper presents a novel deep representation learning method for the efficient detection of arrhythmic beats. To mitigate the issues associated with the imbalanced data distribution, a novel re-sampling strategy is introduced. Unlike the existing oversampling methods, the proposed technique transforms majority-class samples into minority-class samples with a novel translation loss function. This approach assists the model in learning a more generalized representation of crucially important minority class samples. Moreover, by exploiting an auxiliary feature, an augmented attention module is designed that focuses on the most relevant and target-specific information. We adopted an inter-patient classification paradigm to evaluate the proposed method. The experimental results of this study on the MIT-BIH arrhythmia database clearly indicate that the proposed model with augmented attention mechanism and over-sampling strategy significantly learns a balanced deep representation and improves the classification performance of vital heartbeats.

An optimized Bi-LSTM with random synthetic over-sampling strategy for network intrusion detection

An optimized Bi-LSTM with random synthetic over-sampling strategy for network intrusion detection

Smoclust: synthetic minority oversampling based on stream clustering for evolving data streams

Many real-world data stream applications not only suffer from concept drift but also class imbalance. Yet, very few existing studies investigated this joint challenge. Data difficulty factors, which have been shown to be key challenges in class imbalanced data streams, are not taken into account by existing approaches when learning class imbalanced data streams. In this work, we propose a drift adaptable oversampling strategy to synthesise minority class examples based on stream clustering. The motivation is that stream clustering methods continuously update themselves to reflect the characteristics of the current underlying concept, including data difficulty factors. This nature can potentially be used to compress past information without caching data in the memory explicitly. Based on the compressed information, synthetic examples can be created within the region that recently generated new minority class examples. Experiments with artificial and real-world data streams show that the proposed approach can handle concept drift involving different minority class decomposition better than existing approaches, especially when the data stream is severely class imbalanced and presenting high proportions of safe and borderline minority class examples.

A human-interpretable machine learning pipeline based on ultrasound to support leiomyosarcoma diagnosis

The preoperative evaluation of myometrial tumors is essential to avoid delayed treatment and to establish the appropriate surgical approach. Specifically, the differential diagnosis of leiomyosarcoma (LMS) is particularly challenging due to the overlapping of clinical, laboratory and ultrasound features between fibroids and LMS. In this work, we present a human-interpretable machine learning (ML) pipeline to support the preoperative differential diagnosis of LMS from leiomyomas, based on both clinical data and gynecological ultrasound assessment of 68 patients (8 with LMS diagnosis). The pipeline provides the following novel contributions: (i) end-users have been involved both in the definition of the ML tasks and in the evaluation of the overall approach; (ii) clinical specialists get a full understanding of both the decision-making mechanisms of the ML algorithms and the impact of the features on each automatic decision. Moreover, the proposed pipeline addresses some of the problems concerning both the imbalance of the two classes by analyzing and selecting the best combination of the synthetic oversampling strategy of the minority class and the classification algorithm among different choices, and the explainability of the features at global and local levels. The results show very high performance of the best strategy (AUC = 0.99, F1 = 0.87) and the strong and stable impact of two ultrasound-based features (i.e., tumor borders and consistency of the lesions). Furthermore, the SHAP algorithm was exploited to quantify the impact of the features at the local level and a specific module was developed to provide a template-based natural language (NL) translation of the explanations for enhancing their interpretability and fostering the use of ML in the clinical setting.

Generative adversarial minority enlargement—A local linear over-sampling synthetic method

The imbalanced data problem is widely recognized in real-world datasets. To avoid learning bias on imbalanced data, the over-sampling strategy is well studied and adopted for generating synthetic minority samples. Wherein, the Synthetic Minority Over-sampling Technology and its improved algorithms become standard baselines. In recent years, the popular Generative Adversarial Networks and its enhanced variants, introduced from the computer vision community, are believed to generate better samples, by approximating the true data distribution. Yet, we notice that the synthetic samples for the minority category in these existing methods is usually restrained in a limited samples space known in advance, which may mislead the classifiers trained on them to take data in the unsampled region of the minority category as from the majority category. Given such limitations, we propose a Generative Adversarial Minority Enlargement (GAME) method to intentionally extend the sampling margin during data generative and adversarial phases. This is accomplished by adjusting the parameters of a local linear model to approach the majority category. We conduct an extensive evaluation on 28 datasets of different domains, extracted from the UCI real-world datasets. The results show that GAME can achieve more balanced and stable results efficiently than 18 state-of-the-art methods.

Handling imbalanced data in supervised machine learning for lithological mapping using remote sensing and airborne geophysical data

Abstract With balanced training sample (TS) data, learning algorithms offer good results in lithology classification. Meanwhile, unprecedented lithological mapping in remote places is predicted to be difficult, resulting in limited and unbalanced samples. To address this issue, we can use a variety of techniques, including ensemble learning (such as random forest [RF]), over/undersampling, class weight tuning, and hybrid approaches. This work investigates and analyses many strategies for dealing with imbalanced data in lithological classification based on RF algorithms with limited drill log samples using remote sensing and airborne geophysical data. The research was carried out at Komopa, Paniai District, Papua Province, Indonesia. The class weight tuning, oversampling, and balance class weight procedures were used, with TSs ranging from 25 to 500. The oversampling approach outperformed the class weight tuning and balance class weight procedures in general, with the following metric values: 0.70–0.80 (testing accuracy), 0.43–0.56 (F1 score), and 0.32–0.59 (Kappa score). The visual comparison also revealed that the oversampling strategy gave the most reliable classifications: if the imbalance ratio is proportionate to the coverage area in each lithology class, the classifier capability is optimal.

Assessor-guided learning for continual environments

This paper proposes an assessor-guided learning strategy for continual learning where an assessor guides the learning process of a base learner by controlling the direction and pace of the learning process thus allowing an efficient learning of new environments while protecting against the catastrophic interference problem. The assessor is trained in a meta-learning manner with a meta-objective to boost the learning process of the base learner. It performs a soft-weighting mechanism of every sample accepting positive samples while rejecting negative samples. The training objective of a base learner is to minimize a meta-weighted combination of the cross entropy loss function, the dark experience replay (DER) loss function and the knowledge distillation loss function whose interactions are controlled in such a way to attain an improved performance. A compensated over-sampling (COS) strategy is developed to overcome the class imbalanced problem of the episodic memory due to limited memory budgets. Our approach, Assessor-Guided Learning Approach (AGLA), has been evaluated in the class-incremental and task-incremental learning problems. AGLA achieves improved performances compared to its competitors while the theoretical analysis of the COS strategy is offered. Source codes of AGLA, baseline algorithms and experimental logs are shared publicly in https://github.com/anwarmaxsum/AGLA for further study.

An Oversampling Mechanism for Multimajority Datasets using SMOTE and Darwinian Particle Swarm Optimisation

Data skewness continues to be one of the leading factors which adversely impacts the machine learning algorithms performance. An approach to reduce this negative effect of the data variance is to pre-process the former dataset with data level resampling strategies. Resampling strategies have been seen in two forms, oversampling and undersampling. An oversampling strategy is proposed in this article for tackling multiclass imbalanced datasets. This proposed approach optimises the state-of-the-art oversampling technique SMOTE with the Darwinian Particle Swarm Optimization technique. This proposed method DOSMOTE generates synthetic optimised samples for balancing the datasets. This strategy will be more effective on multimajority datasets. An experimental study is performed on peculiar multimajority datasets to measure the effectiveness of the proposed approach. As a result, the proposed method produces promising results when compared to the conventional oversampling strategies.

Identify essential genes based on clustering based synthetic minority oversampling technique

Prediction of essential genes in a life organism is one of the central tasks in synthetic biology. Computational predictors are desired because experimental data is often unavailable. Recently, some sequence-based predictors have been constructed to identify essential genes. However, their predictive performance should be further improved. One key problem is how to effectively extract the sequence-based features, which are able to discriminate the essential genes. Another problem is the imbalanced training set. The amount of essential genes in human cell lines is lower than that of non-essential genes. Therefore, predictors trained with such imbalanced training set tend to identify an unseen sequence as a non-essential gene. Here, a new over-sampling strategy was proposed called Clustering based Synthetic Minority Oversampling Technique (CSMOTE) to overcome the imbalanced data issue. Combining CSMOTE with the Z curve, the global features, and Support Vector Machines, a new protocol called iEsGene-CSMOTE was proposed to identify essential genes. The rigorous jackknife cross validation results indicated that iEsGene-CSMOTE is better than the other competing methods. The proposed method outperformed λ-interval Z curve by 35.48% and 11.25% in terms of Sn and BACC, respectively.

Imbalanced Multimodal Attention-Based System for Multiclass House Price Prediction

House price prediction is an important problem for individuals, companies, organizations, and governments. With a vast amount of diversified and multimodal data available about houses, the predictive models built should seek to make the best use of these data. This leads to the complex problem of how to effectively use multimodal data for house price prediction. Moreover, this is also a context suffering from class imbalance, an issue that cannot be disregarded. In this paper, we propose a new algorithm for addressing these problems: the imbalanced multimodal attention-based system (IMAS). The IMAS makes use of an oversampling strategy that operates on multimodal data, namely using text, numeric, categorical, and boolean data types. A self-attention mechanism is embedded to leverage the usage of neighboring information that can benefit the model’s performance. Moreover, the self-attention mechanism allows for the determination of the features that are the most relevant and adapts the weights used according to that information when performing inference. Our experimental results show the clear advantage of the IMAS, which outperforms all the competitors tested. The analysis of the weights obtained through the self-attention mechanism provides insights into the features’ relevance and also supports the importance of using this mechanism in the predictive model.

Machine learning for the prediction of minor amputation in University of Texas grade 3 diabetic foot ulcers.

Minor amputations are performed in a large proportion of patients with diabetic foot ulcers (DFU) and early identification of the outcome of minor amputations facilitates medical decision-making and ultimately reduces major amputations and deaths. However, there are currently no clinical predictive tools for minor amputations in patients with DFU. We aim to establish a predictive model based on machine learning to quickly identify patients requiring minor amputation among newly admitted patients with DFU. Overall, 362 cases with University of Texas grade (UT) 3 DFU were screened from tertiary care hospitals in East China. We utilized the synthetic minority oversampling strategy to compensate for the disparity in the initial dataset. A univariable analysis revealed nine variables to be included in the model: random blood glucose, years with diabetes, cardiovascular diseases, peripheral arterial diseases, DFU history, smoking history, albumin, creatinine, and C-reactive protein. Then, risk prediction models based on five machine learning algorithms: decision tree, random forest, logistic regression, support vector machine, and extreme gradient boosting (XGBoost) were independently developed with these variables. After evaluation, XGBoost earned the highest score (accuracy 0.814, precision 0.846, recall 0.767, F1-score 0.805, and AUC 0.881). For convenience, a web-based calculator based on our data and the XGBoost algorithm was established (https://dfuprediction.azurewebsites.net/). These findings imply that XGBoost can be used to develop a reliable prediction model for minor amputations in patients with UT3 DFU, and that our online calculator will make it easier for clinicians to assess the risk of minor amputations and make proactive decisions.

SORAG: Synthetic Data Over-Sampling Strategy on Multi-Label Graphs

In many real-world networks of interest in the field of remote sensing (e.g., public transport networks), nodes are associated with multiple labels, and node classes are imbalanced; that is, some classes have significantly fewer samples than others. However, the research problem of imbalanced multi-label graph node classification remains unexplored. This non-trivial task challenges the existing graph neural networks (GNNs) because the majority class can dominate the loss functions of GNNs and result in the overfitting of the majority class features and label correlations. On non-graph data, minority over-sampling methods (such as the synthetic minority over-sampling technique and its variants) have been demonstrated to be effective for the imbalanced data classification problem. This study proposes and validates a new hypothesis with unlabeled data over-sampling, which is meaningless for imbalanced non-graph data; however, feature propagation and topological interplay mechanisms between graph nodes can facilitate the representation learning of imbalanced graphs. Furthermore, we determine empirically that ensemble data synthesis through the creation of virtual minority samples in the central region of a minority and generation of virtual unlabeled samples in the boundary region between a minority and majority is the best practice for the imbalanced multi-label graph node classification task. Our proposed novel data over-sampling framework is evaluated using multiple real-world network datasets, and it outperforms diverse, strong benchmark models by a large margin.

NH-PINN: Neural homogenization-based physics-informed neural network for multiscale problems

Physics-informed neural network (PINN) is a data-driven approach to solving equations. It is successful in many applications; however, the accuracy of the PINN is not satisfactory when it is used to solve multiscale equations. Homogenization approximates a multiscale equation by a homogenized equation without multiscale property; it includes solving cell problems and the homogenized equation. The cell problems are periodic, and we propose an oversampling strategy that significantly improves the PINN accuracy on periodic problems. The homogenized equation has a constant or slow dependency coefficient and can also be solved by PINN accurately. We hence proposed a 3-step method, neural homogenization based PINN (NH-PINN), to improve the PINN accuracy for solving multiscale problems with the help of homogenization.

An Online Prediction Approach Based on Incremental Support Vector Machine for Dynamic Multiobjective Optimization

Real-world multiobjective optimization problems usually involve conflicting objectives that change over time, which requires the optimization algorithms to quickly track the Pareto-optimal front (POF) when the environment changes. In recent years, evolutionary algorithms based on prediction models have been considered promising. However, most existing approaches only make predictions based on the linear correlation between a finite number of optimal solutions in two or three previous environments. These incomplete information extraction strategies may lead to low prediction accuracy in some instances. In this article, an incremental support vector machine (ISVM)-based dynamic multiobjective evolutionary algorithm, in short called ISVM-DMOEA, is proposed. We treat the solving of dynamic multiobjective optimization problems (DMOPs) as an online learning process, using the continuously obtained optimal solution to update an ISVM without discarding the solution information at earlier time. ISVM is then used to filter random solutions and generate an initial population for the next moment. To overcome the obstacle of insufficient training samples, a synthetic minority oversampling strategy is implemented before the training of ISVM. The advantage of this approach is that the nonlinear correlation between solutions can be explored online by ISVM, and the information contained in all historical optimal solutions can be exploited to a greater extent. The experimental results and comparison with the chosen state-of-the-art algorithms demonstrate that the proposed algorithm can effectively tackle DMOPs.

Risk Prediction of Major Adverse Cardiovascular Events Occurrence Within 6 Months After Coronary Revascularization: Machine Learning Study

BackgroundAs a major health hazard, the incidence of coronary heart disease has been increasing year by year. Although coronary revascularization, mainly percutaneous coronary intervention, has played an important role in the treatment of coronary heart disease, major adverse cardiovascular events (MACE) such as recurrent or persistent angina pectoris after coronary revascularization remain a very difficult problem in clinical practice.ObjectiveGiven the high probability of MACE after coronary revascularization, the aim of this study was to develop and validate a predictive model for MACE occurrence within 6 months based on machine learning algorithms.MethodsA retrospective study was performed including 1004 patients who had undergone coronary revascularization at The People’s Hospital of Liaoning Province and Affiliated Hospital of Liaoning University of Traditional Chinese Medicine from June 2019 to December 2020. According to the characteristics of available data, an oversampling strategy was adopted for initial preprocessing. We then employed six machine learning algorithms, including decision tree, random forest, logistic regression, naïve Bayes, support vector machine, and extreme gradient boosting (XGBoost), to develop prediction models for MACE depending on clinical information and 6-month follow-up information. Among all samples, 70% were randomly selected for training and the remaining 30% were used for model validation. Model performance was assessed based on accuracy, precision, recall, F1-score, confusion matrix, area under the receiver operating characteristic (ROC) curve (AUC), and visualization of the ROC curve.ResultsUnivariate analysis showed that 21 patient characteristic variables were statistically significant (P<.05) between the groups without and with MACE. Coupled with these significant factors, among the six machine learning algorithms, XGBoost stood out with an accuracy of 0.7788, precision of 0.8058, recall of 0.7345, F1-score of 0.7685, and AUC of 0.8599. Further exploration of the models to identify factors affecting the occurrence of MACE revealed that use of anticoagulant drugs and course of the disease consistently ranked in the top two predictive factors in three developed models.ConclusionsThe machine learning risk models constructed in this study can achieve acceptable performance of MACE prediction, with XGBoost performing the best, providing a valuable reference for pointed intervention and clinical decision-making in MACE prevention.

Lung cancer diagnosis using deep attention-based multiple instance learning and radiomics.

BackgroundEarly diagnosis of lung cancer is a key intervention for the treatment of lung cancer in which computer‐aided diagnosis (CAD) can play a crucial role. Most published CAD methods perform lung cancer diagnosis by classifying each lung nodule in isolation. However, this does not reflect clinical practice, where clinicians diagnose a patient based on a set of images of nodules, instead of looking at one nodule at a time. Besides, the low interpretability of the output provided by these methods presents an important barrier for their adoption.MethodIn this article, we treat lung cancer diagnosis as a multiple instance learning (MIL) problem, which better reflects the diagnosis process in the clinical setting and provides higher interpretability of the output. We selected radiomics as the source of input features and deep attention‐based MIL as the classification algorithm. The attention mechanism provides higher interpretability by estimating the importance of each instance in the set for the final diagnosis. To improve the model's performance in a small imbalanced dataset, we propose a new bag simulation method for MIL.Results and conclusionThe results show that our method can achieve a mean accuracy of 0.807 with a standard error of the mean (SEM) of 0.069, a recall of 0.870 (SEM 0.061), a positive predictive value of 0.928 (SEM 0.078), a negative predictive value of 0.591 (SEM 0.155), and an area under the curve (AUC) of 0.842 (SEM 0.074), outperforming other MIL methods. Additional experiments show that the proposed oversampling strategy significantly improves the model's performance. In addition, experiments show that our method provides a good indication of the importance of each nodule in determining the diagnosis, which combined with the well‐defined radiomic features, to make the results more interpretable and acceptable for doctors and patients.

Bicriteria Oversampling for Imbalanced Data Classification

The paper proposes bicriteria oversampling strategy for mining imbalanced data. We use two specialized criteria for oversampling -classification potential and distance from the borderline between minority and majority instances. The potential is to be maximized and the distance minimized. The required number of synthetic examples is selected from the non-dominated set of examples produced by the evolutionary algorithm. At the final step the balanced set of examples is used by GEP classifier. Computational experiment confirmed that the approach assures high quality performance.

Construction of Mitochondrial Protection and Monitoring Model of Lon Protease Based on Machine Learning under Myocardial Ischemia Environment.

The localization of a protein's submitochondrial structure is important for therapeutic design of associated disorders caused by mitochondrial abnormalities because many human diseases are directly tied to mitochondria. When Lon protease expression changes, glycolysis replaces respiratory metabolism in the cell, which is a common occurrence in cancer cells. The fact that protein formation is a dynamic research object makes it impossible to reproduce the unique living environment of proteins in an experimental setting, which surely makes it more challenging to determine protein function through experiments. This research suggests a model of Lon protease-based mitochondrial protection under myocardial ischemia based on ML (machine learning). To ensure the balance of all submitochondrial proteins, the data set is processed using a random oversampling method, each overlapping fixed-length subsequence that is created from the protein sequence functions as a channel in the convolution layer. The results demonstrate that applying the oversampling strategy increases the ROC value by 17.6%-21.3%. Our prediction method is successful as evidenced by the fact that ML prediction outperforms the predictions of other conventional classifiers.

Prediction on Domestic Violence in Bangladesh during the COVID-19 Outbreak Using Machine Learning Methods

The COVID-19 outbreak resulted in preventative measures and restrictions for Bangladesh during the summer of 2020—these unstable and stressful times led to multiple social problems (e.g., domestic violence and divorce). Globally, researchers, policymakers, governments, and civil societies have been concerned about the increase in domestic violence against women and children during the ongoing COVID-19 pandemic. In Bangladesh, domestic violence against women and children has increased during the COVID-19 pandemic. In this article, we investigated family violence among 511 families during the COVID-19 outbreak. Participants were given questionnaires to answer, for a period of over ten days; we predicted family violence using a machine learning-based model. To predict domestic violence from our data set, we applied random forest, logistic regression, and Naive Bayes machine learning algorithms to our model. We employed an oversampling strategy named the Synthetic Minority Oversampling Technique (SMOTE) and the chi-squared statistical test to, respectively, solve the imbalance problem and discover the feature importance of our data set. The performances of the machine learning algorithms were evaluated based on accuracy, precision, recall, and F-score criteria. Finally, the receiver operating characteristic (ROC) and confusion matrices were developed and analyzed for three algorithms. On average, our model, with the random forest, logistic regression, and Naive Bayes algorithms, predicted family violence with 77%, 69%, and 62% accuracy for our data set. The findings of this study indicate that domestic violence has increased and is highly related to two features: family income level during the COVID-19 pandemic and education level of the family members.