Imbalanced Training Data Research Articles

Identification of Type VI Effector Proteins Using a Novel Ensemble Classifier

The type VI secretion system (T6SS) delivers effector proteins (Type VI secretion system effectors, termed T6SEs) into neighboring target cells. Many human pathogens express T6SEs, including Vibrio cholera, Burkholderia spp., and Pseudomonas aeruginosa. T6SEs play vital roles in the competitive survival and pathogenesis of bacterial populations. Several machine-learning methods are able to distinguish T6SEs from non-T6SEs. However, we believe there is room for further development. Therefore, herein we propose a more powerful ensemble predictor for identifying T6SEs. Initially, we construct a benchmark dataset from existing studies and databases. Then we use $k$ -separated-bigrams-PSSM (a type of feature encoding) to convert the protein sequences to mathematical vectors. A synthetic minority oversampling technique (SMOTE) is next employed to solve the training data imbalance problem. Finally, we employ a soft voting strategy to construct an integrated model combining six fine-tuned base classifiers. The model we propose performs excellently in terms of accuracy (ACC, 99.0%), Matthew's correlation coefficient (MCC, 97.8%), sensitivity (SN, 97.1%), and specificity (SP, 100%) in independent testing.

GT-GAN: A General Transductive Zero-Shot Learning Method Based on GAN

Most zero-shot learning (ZSL) methods based on generative adversarial networks (GANs) utilize random noise and semantic descriptions to synthesize image features of unseen classes, which alleviates the problem of training data imbalance between seen and unseen classes. However, these methods usually only learn the distributions of seen classes in the training stage, ignoring the unseen ones. Due to the different distributions of seen and unseen samples, i.e., image features, these methods cannot generate unseen features of sufficient quality, so the performances are also limited, especially for the generalized zero-shot learning (GZSL) setting. In this article, we propose a general transductive method based on GANs, called GT-GAN, which can improve the quality of generated unseen image features and therefore benefit the classification. A new loss function is introduced to make the relative positions between each unseen image and its $k$ nearest neighbors in the feature space as consistent as possible with their relative positions in the semantic space; this loss function may be easily applied in most existing GAN-based models. Experimental results on five benchmark datasets show a significant improvement in accuracy compared with that of original models, especially in the GZSL setting.

Machinery fault diagnosis with imbalanced data using deep generative adversarial networks

Despite the recent advances of intelligent data-driven fault diagnosis methods on rotating machines, balanced training data for different machine health conditions are assumed in most studies. However, the signals in machine faulty states are usually difficult and expensive to collect, resulting in imbalanced training dataset in most cases. That significantly deteriorates the effectiveness of the existing data-driven approaches. This paper proposes a deep learning-based fault diagnosis method to address the imbalanced data problem by explicitly creating additional training data. Generative adversarial networks are firstly used to learn the mapping between the distributions of noise and real machinery temporal vibration data, and additional realistic fake samples can be generated to balance and further expand the available dataset afterwards. Through experiments on two rotating machinery datasets, it is validated that the data-driven methods can significantly benefit from the data augmentation, and the proposed method offers a promising tool on fault diagnosis with imbalanced training data.

Detection of Wildfire Smoke Images Based on a Densely Dilated Convolutional Network

Recently, many researchers have attempted to use convolutional neural networks (CNNs) for wildfire smoke detection. However, the application of CNNs in wildfire smoke detection still faces several issues, e.g., the high false-alarm rate of detection and the imbalance of training data. To address these issues, we propose a novel framework integrating conventional methods into CNN for wildfire smoke detection, which consisted of a candidate smoke region segmentation strategy and an advanced network architecture, namely wildfire smoke dilated DenseNet (WSDD-Net). Candidate smoke region segmentation removed the complex backgrounds of the wildfire smoke images. The proposed WSDD-Net achieved multi-scale feature extraction by combining dilated convolutions with dense block. In order to solve the problem of the dataset imbalance, an improved cross entropy loss function, namely balanced cross entropy (BCE), was used instead of the original cross entropy loss function in the training process. The proposed WSDD-Net was evaluated according to two smoke datasets, i.e., WS and Yuan, and achieved a high AR (99.20%) and a low FAR (0.24%). The experimental results demonstrated that the proposed framework had better detection capabilities under different negative sample interferences.

Recognizing roles of online illegal gambling participants: An ensemble learning approach

Online gambling has exploded into a substantial global industry over the past two decades. Despite its prosperity, online gambling is explicitly prohibited or restricted in most countries due to social problems it caused. This has given rise to black markets of illegal online gambling (IOG) which are replete with criminal potential. To help criminal investigators find key members of an IOG organization and fight against them, in this study, we propose an ensemble learning approach named EC4PRR to automatically identify the roles that IOG participants play in their ecosystem. We extract two categories of features, i.e., transaction statistical features (TSF) and network structural features (NSF), from monetary transaction data effectively represent participants. Since the number of participants in different roles is typically unbalanced, we train the classifier with a combination of under-sampling and ensemble learning strategies to reduce the impact of imbalanced training data. Experiments were carried out with real-world case data. Experimental results demonstrate the validity of the proposed features as well as the feasibility of EC4PRR.

Dual Adversarial Transfer for Sequence Labeling.

We propose a new architecture for addressing sequence labeling, termed Dual Adversarial Transfer Network (DATNet). Specifically, the proposed DATNet includes two variants, i.e., DATNet-F and DATNet-P, which are proposed to explore effective feature fusion between high and low resource. To address the noisy and imbalanced training data, we propose a novel Generalized Resource-Adversarial Discriminator (GRAD) and adopt adversarial training to boost model generalization. We investigate the effects of different components of DATNet across different domains and languages, and show that significant improvement can be obtained especially for low-resource data. Without augmenting any additional hand-crafted features, we achieve state-of-the-art performances on CoNLL, Twitter, PTB-WSJ, OntoNotes and Universal Dependencies with three popular sequence labeling tasks, i.e., Named entity recognition (NER), Part-of-Speech (POS) Tagging and Chunking.

Efficiency of Extreme Gradient Boosting for Imbalanced Land Cover Classification Using an Extended Margin and Disagreement Performance

Imbalanced learning is a methodological challenge in remote sensing communities, especially in complex areas where the spectral similarity exists between land covers. Obtaining high-confidence classification results for imbalanced class issues is highly important in practice. In this paper, extreme gradient boosting (XGB), a novel tree-based ensemble system, is employed to classify the land cover types in Very-high resolution (VHR) images with imbalanced training data. We introduce an extended margin criterion and disagreement performance to evaluate the efficiency of XGB in imbalanced learning situations and examine the effect of minority class spectral separability on model performance. The results suggest that the uncertainty of XGB associated with correct classification is stable. The average probability-based margin of correct classification provided by XGB is 0.82, which is about 46.30% higher than that by random forest (RF) method (0.56). Moreover, the performance uncertainty of XGB is insensitive to spectral separability after the sample imbalance reached a certain level (minority:majority > 10:100). The impact of sample imbalance on the minority class is also related to its spectral separability, and XGB performs better than RF in terms of user accuracy for the minority class with imperfect separability. The disagreement components of XGB are better and more stable than RF with imbalanced samples, especially for complex areas with more types. In addition, appropriate sample imbalance helps to improve the trade-off between the recognition accuracy of XGB and the sample cost. According to our analysis, this margin-based uncertainty assessment and disagreement performance can help users identify the confidence level and error component in similar classification performance (overall, producer, and user accuracies).

Copula Ordinal Regression Framework for Joint Estimation of Facial Action Unit Intensity

Joint modeling of the intensity of multiple facial action units (AUs) from face images is challenging due to the large number of AUs (30+) and their intensity levels (6). This is in part due to the lack of suitable models that can efficiently handle such a large number of outputs/classes simultaneously, but also due to the lack of suitable data the models on. For this reason, majority of the methods resort to independent classifiers for the AU intensity. This is suboptimal for at least two reasons: the facial appearance of some AUs changes depending on the intensity of other AUs, and some AUs co-occur more often than others. To this end, we propose the Copula regression approach for modeling multivariate ordinal variables. Our model accounts for $ordinal$ordinal structure in output variables and their $non$non-$linear$linear dependencies via copula functions modeled as cliques of a conditional random fields. The copula ordinal regression model achieves the joint learning and inference of intensities of multiple AUs, while being computationally tractable. We demonstrate the effectiveness of our approach on three challenging datasets of naturalistic facial expressions and we show that the estimation of target AU intensities improves especially in the case of (a) noisy image features, (b) head-pose variations and (c) imbalanced training data. Lastly, we show that the proposed approach consistently outperforms (i) independent modeling of AU intensities and (ii) the state-of-the-art approach for the target task and (iii) deep convolutional neural networks.

Kernel modified optimal margin distribution machine for imbalanced data classification

Although the optimal margin distribution machine (ODM) has better generalization performance in pattern recognition than traditional classifiers, ODM as well as traditional classifiers often suffers from data imbalance. To address this, this paper proposes a kernel modified ODM (KMODM) to eliminate the side effect of imbalanced data. According to the mechanism of ODM, a novel conformal function is designed to scale the kernel matrix of ODM, this can increases the separability of the training data in the feature space. In addition, to eliminate the skew of the separator toward minority class, KMODM introduces two free parameters in conformal function to balance the influence of different training data on separating hyperplane. Experimental results on two-dimensional visualization data show that KMODM can alleviate the skew of the separating hyperplane caused by imbalanced data. For most of ten UCI data sets, KMODM can broad the margin of the minority class and achieve the highest average G-mean and F1 score. This means that KMODM has more balanced detection rate and better generalization performance compared to other baseline methods, especially in presence of heavily imbalanced training data.

Identifying mortality risk factors amongst acute coronary syndrome patients admitted to Arabian Gulf hospitals using machine‐learning methods

AbstractAcute coronary syndrome (ACS) is a leading cause of mortality and morbidity in the Arabian Gulf. In this study, the in‐hospital mortality amongst patients admitted with ACS to Arabian Gulf hospitals is predicted using a comprehensive modelling framework that combines powerful machine‐learning methods such as support‐vector machine (SVM), Naïve Bayes (NB), artificial neural networks (NN), and decision trees (DT). The performance of the machine‐learning methods is compared with that of the performance of a commonly used statistical method, namely, logistic regression (LR). The study follows the current practise of computing mortality risk using risk scores such as the Global Registry of Acute Coronary Events (GRACE) score, which has not been validated for Arabian Gulf patients. Cardiac registry data of 7,000 patients from 65 hospitals located in Arabian Gulf countries are used for the study. This study is unique as it uses a contemporary data analytics framework. A k‐fold (k = 10) cross‐validation is utilized to generate training and validation samples from the GRACE dataset. The machine‐learning‐based predictive models often incur prejudgments for imbalanced training data patterns. To mitigate the data imbalance due to scarce observations for in‐hospital mortalities, we have utilized specialized methods such as random undersampling (RUS) and synthetic minority over sampling technique (SMOTE). A detailed simulation experimentation is carried out to build models with each of the five predictive methods (LR, NN, NB, SVM, and DT) for the each of the three datasets k‐fold subsamples generated. The predictive models are developed under three schemes of the k‐fold samples that include no data imbalance, RUS, and SMOTE. We have implemented an information fusion method rooted in computing weighted impact scores obtain for an individual medical history attributes from each of the predictive models simulated for a collective recommendation based on an impact score specific to a predictor. Finally, we grouped the predictors using fuzzy c‐mean clustering method into three categories, high‐, medium‐, and low‐risk factors for in‐hospital mortality due to ACS. Our study revealed that patients with medical history related to the presences of peripheral artery disease, congestive heart failure, cardiovascular transient ischemic attack valvular disease, and coronary artery bypass grafting amongst others have the most risk for in‐hospital mortality.

Neural Network Classifier-Based OPC With Imbalanced Training Data

Machine learning-guided optical proximity correction, called ML-OPC in this paper, has recently been proposed to alleviate long runtime of model-based OPC. ML-OPC using regression methods has been presented but with limited prediction accuracy. We propose neural network classifier-based OPC (NNC-OPC), in which a neural network classifier serves as a mask bias model. A few techniques are applied to enhance basic NNC-OPC: parameterization of layout segments using polar Fourier transform signals, dimensionality reduction through weighted principal component analysis, and sampling of training layout segments. Training segments are typically imbalanced over the range of mask biases, which may cause large prediction error for segments that appear less frequently. This is resolved by three techniques: 1) synthetic data generation; 2) class reorganization; and 3) an adaptive learning rate. Experiments with NNC-OPC with all techniques applied indicate that prediction error of mask bias and training time are reduced by 29% and 80%, respectively, compared to state-of-the-art ML-OPC with regression methods.

Integrating Growth and Environmental Parameters to Discriminate Powdery Mildew and Aphid of Winter Wheat Using Bi-Temporal Landsat-8 Imagery

Monitoring and discriminating co-epidemic diseases and pests at regional scales are of practical importance in guiding differential treatment. A combination of vegetation and environmental parameters could improve the accuracy for discriminating crop diseases and pests. Different diseases and pests could cause similar stresses and symptoms during the same crop growth period, so combining growth period information can be useful for discerning different changes in crop diseases and pests. Additionally, problems associated with imbalanced data often have detrimental effects on the performance of image classification. In this study, we developed an approach for discriminating crop diseases and pests based on bi-temporal Landsat-8 satellite imagery integrating both crop growth and environmental parameters. As a case study, the approach was applied to data during a period of typical co-epidemic outbreak of winter wheat powdery mildew and aphids in the Shijiazhuang area of Hebei Province, China. Firstly, bi-temporal remotely sensed features characterizing growth indices and environmental factors were calculated based on two Landsat-8 images. The synthetic minority oversampling technique (SMOTE) algorithm was used to resample the imbalanced training data set before model construction. Then, a back propagation neural network (BPNN) based on a new training data set balanced by the SMOTE approach (SMOTE-BPNN) was developed to generate the regional wheat disease and pest distribution maps. The original training data set-based BPNN and support vector machine (SVM) methods were used for comparison and testing of the initial results. Our findings suggest that the proposed approach incorporating both growth and environmental parameters of different crop periods could distinguish wheat powdery mildew and aphids at the regional scale. The bi-temporal growth indices and environmental factors-based SMOTE-BPNN, BPNN, and SVM models all had an overall accuracy high than 80%. Meanwhile, the SMOTE-BPNN method had the highest G-means among the three methods. These results revealed that the combination of bi-temporal crop growth and environmental parameters is essential for improving the accuracy of the crop disease and pest discriminating models. The combination of SMOTE and BPNN could effectively improve the discrimination accuracy of the minor disease or pest.

Domain-specific word embeddings for patent classification

PurposePatent offices and other stakeholders in the patent domain need to classify patent applications according to a standardized classification scheme. The purpose of this paper is to examine the novelty of an application it can then be compared to previously granted patents in the same class. Automatic classification would be highly beneficial, because of the large volume of patents and the domain-specific knowledge needed to accomplish this costly manual task. However, a challenge for the automation is patent-specific language use, such as special vocabulary and phrases.Design/methodology/approachTo account for this language use, the authors present domain-specific pre-trained word embeddings for the patent domain. The authors train the model on a very large data set of more than 5m patents and evaluate it at the task of patent classification. To this end, the authors propose a deep learning approach based on gated recurrent units for automatic patent classification built on the trained word embeddings.FindingsExperiments on a standardized evaluation data set show that the approach increases average precision for patent classification by 17 percent compared to state-of-the-art approaches. In this paper, the authors further investigate the model’s strengths and weaknesses. An extensive error analysis reveals that the learned embeddings indeed mirror patent-specific language use. The imbalanced training data and underrepresented classes are the most difficult remaining challenge.Originality/valueThe proposed approach fulfills the need for domain-specific word embeddings for downstream tasks in the patent domain, such as patent classification or patent analysis.

Improving AdaBoost-based Intrusion Detection System (IDS) Performance on CIC IDS 2017 Dataset

This paper considers the use of Synthetic Minority Oversampling Technique (SMOTE), Principal Component Analysis (PCA), and Ensemble Feature Selection (EFS) to improve the performance of AdaBoost-based Intrusion Detection System (IDS) on the latest and challenging CIC IDS 2017 Dataset [1]. Previous research [1] has proposed the use of AdaBoost classifier to cope with the new dataset. However, due to several problems such as imbalance of training data and inappropriate selection of classification methods, the performance is still inferior. In this research, we aim at constructing an improvement performance intrusion detection approach to handle the imbalance of training data, SMOTE is selected to tackle the problem. Moreover, Principal Component Analysis (PCA) and Ensemble Feature Selection (EFS) are applied as the feature selection to select important attributes from the new dataset. The evaluation results show that the proposed AdaBoost classifier using PCA and SMOTE yields Area Under the Receiver Operating Characteristic curve (AUROC) of 92% and the AdaBoost classifier using EFS and SMOTE produces an accuracy, precision, recall, and F1 Score of 81.83 %, 81.83%, 100%, and 90.01% respectively.

Recurrent generative adversarial network for learning imbalanced medical image semantic segmentation

We propose a new recurrent generative adversarial architecture named RNN-GAN to mitigate imbalance data problem in medical image semantic segmentation where the number of pixels belongs to the desired object are significantly lower than those belonging to the background. A model trained with imbalanced data tends to bias towards healthy data which is not desired in clinical applications and predicted outputs by these networks have high precision and low recall. To mitigate imbalanced training data impact, we train RNN-GAN with proposed complementary segmentation mask, in addition, ordinary segmentation masks. The RNN-GAN consists of two components: a generator and a discriminator. The generator is trained on the sequence of medical images to learn corresponding segmentation label map plus proposed complementary label both at a pixel level, while the discriminator is trained to distinguish a segmentation image coming from the ground truth or from the generator network. Both generator and discriminator substituted with bidirectional LSTM units to enhance temporal consistency and get inter and intra-slice representation of the features. We show evidence that the proposed framework is applicable to different types of medical images of varied sizes. In our experiments on ACDC-2017, HVSMR-2016, and LiTS-2017 benchmarks we find consistently improved results, demonstrating the efficacy of our approach.

Facial Attribute Recognition With Feature Decoupling and Graph Convolutional Networks

The objective of facial attribute recognition is to predict a set of labels for each face image. Similar to general multi-label image classification task, facial attribute recognition suffers from several typical problems: discriminative feature learning, handling the correlations among attributes, and imbalanced training data. In this paper, we propose a unified facial attribute recognition solution via feature decoupling and graph convolutional networks (GCN): 1) We utilize the orthonormal regularizer to constrain that each dimension of the general facial representation represents a certain visual pattern. Meanwhile, a learnable matrix is unveiled, which can convert general facial feature into attribute-specific representation through modeling the importance of different visual patterns. 2) To handle the correlations between facial attributes, we build a GCN to capture the label dependencies and map the nodes in the proposed GCN to a set of inter-dependent attribute classifiers. Besides, we normalize the weights in all classifiers to alleviate the influence of data imbalance. We have conducted extensive experiments on two benchmarks, and both the qualitative and quantitative evaluation results have demonstrated the effectiveness of the proposed method.

Adversarial unseen visual feature synthesis for Zero-shot Learning

Due to the extreme imbalance of training data between seen classes and unseen classes, most existing methods fail to achieve satisfactory results in the challenging task of Zero-shot Learning (ZSL). To avoid the need for labelled data of unseen classes, in this paper, we investigate how to synthesize visual features for ZSL problem. The key challenge is how to capture the realistic feature distribution of unseen classes without training samples. To this end, we propose a hybrid model consists of Random Attribute Selection (RAS) and conditional Generative Adversarial Network (cGAN). RAS aims to learn the realistic generation of attributes by their correlations in nature. To improve the discrimination for the large number of classes, we add a reconstruction loss in the generative network, which can solve the domain shift problem and significantly improve the classification accuracy. Extensive experiments on four benchmarks demonstrate that our method can outperform all the state-of-the-art methods. Qualitative results show that, compared to conventional generative models, our method can capture more realistic distribution and remarkably improve the variability of the synthesized data.

A hierarchical multi-label classification method based on neural networks for gene function prediction

abstractGene function prediction is used to assign biological or biochemical functions to genes, which continues to be a challenging problem in modern biology. Genes may exhibit many functions simultaneously, and these functions are organized into a hierarchy, such as a directed acyclic graph (DAG) for Gene Ontology (GO). Because of these characteristics, gene function prediction can be seen as a typical hierarchical multi-label classification (HMC) task. A novel HMC method based on neural networks is proposed in this article for predicting gene function based on GO. The proposed method belongs to a local approach by transferring the HMC task to a set of subtasks. There are three strategies implemented in this method to improve its performance. First, to tackle the imbalanced data set problem when building the training data set for each class, negative instances selecting policy and SMOTE approach are used to preprocess each imbalanced training data set. Second, a particular multi-layer perceptron (MLP) is designed for each node in GO. Third, a post processing method based on the Bayesian network is used to guarantee that the results are consistent with the hierarchy constraint. The experimental results indicate that the proposed HMC-MLPN method is a promising method for gene function prediction based on a comparison with two other state-of-the-art methods.

Simultaneous Volumetric Segmentation of Vertebral Bodies and Intervertebral Discs on Fat-Water MR Images.

Fat-water magnetic resonance (MR) images allow automated noninvasive analysis of morphological properties and fat fractions of vertebral bodies (VBs) and intervertebral discs (IVDs) that constitute an important part of human biomechanical systems. In this paper, we propose a fully automated approach for simultaneously segmenting multiple VBs and IVDs on fat-water MR images without prior localization or geometry estimation. This method involved a hierarchical random forest (HRF) classifier and a hierarchical conditional random field (HCRF) that encoded a multi-resolution image pyramid based on a set of multiscale local and contextual features. The HRF classifier employed penalized multivariate linear discriminants and SMOTEBagging to handle limited and imbalanced training data with large feature dimension. The HCRF estimated optimum labels according to their spatial and hierarchical consistencies by using the layer-wise significant features determined over the trained HRF classifier. To handle variable sample numbers at different resolutions, resolution-specific hyperparameters were used. This method was trained and evaluated for segmenting 15 thoracic and lumbar VBs and their IVDs on fat-water MR images of a subset of a large cohort data set. It was further evaluated for segmenting seven IVDs of the lower spine on fat-water images of a public grand challenge. These evaluations revealed the comparable accuracy of this method with the state-of-the-art while requiring less computational burden due to a simultaneous localization and segmentation.

The impact of imbalanced training data on machine learning for author name disambiguation

In supervised machine learning for author name disambiguation, negative training data are often dominantly larger than positive training data. This paper examines how the ratios of negative to positive training data can affect the performance of machine learning algorithms to disambiguate author names in bibliographic records. On multiple labeled datasets, three classifiers - Logistic Regression, Na\"ive Bayes, and Random Forest - are trained through representative features such as coauthor names, and title words extracted from the same training data but with various positive-negative training data ratios. Results show that increasing negative training data can improve disambiguation performance but with a few percent of performance gains and sometimes degrade it. Logistic Regression and Na\"ive Bayes learn optimal disambiguation models even with a base ratio (1:1) of positive and negative training data. Also, the performance improvement by Random Forest tends to quickly saturate roughly after 1:10 ~ 1:15. These findings imply that contrary to the common practice using all training data, name disambiguation algorithms can be trained using part of negative training data without degrading much disambiguation performance while increasing computational efficiency. This study calls for more attention from author name disambiguation scholars to methods for machine learning from imbalanced data.