Unbalanced Training Data Research Articles

Rapid discrimination between deleterious and benign missense mutations in the CAGI 6 experiment

We describe the machine learning tool that we applied in the CAGI 6 experiment to predict whether single residue mutations in proteins are deleterious or benign. This tool was trained using only single sequences, i.e., without multiple sequence alignments or structural information. Instead, we used global characterizations of the protein sequence. Training and testing data for human gene mutations was obtained from ClinVar (ncbi.nlm.nih.gov/pub/ClinVar/), and for non-human gene mutations from Uniprot (www.uniprot.org). Testing was done on post-training data from ClinVar. This testing yielded high AUC and Matthews correlation coefficient (MCC) for well trained examples but low generalizability. For genes with either sparse or unbalanced training data, the prediction accuracy is poor. The resulting prediction server is available online at http://www.mamiris.com/Shoni.cagi6.

Towards Robust Colour Texture Analysis with Limited Training Data

Texture analysis plays an important role in different domains of healthcare, agriculture, and industry, where multi-channel sensors are gaining more attention. This contribution presents an interpretable and efficient framework for texture classification and segmentation that exploits colour or channel information and does not require much data to produce accurate results. This makes such a framework well-suited for medical applications and resource-limited hardware. Our approach builds upon a distance-based generalized matrix learning vector quantization (GMLVQ) algorithm. We extend it with parametrized angle-based dissimilarity and introduce a special matrix format for multi-channel images. Classification accuracy evaluation of various model designs was performed on VisTex and ALOT data, and the segmentation application was demonstrated on an agricultural data set. Our extension of parametrized angle dissimilarity measure leads to better model generalization and robustness against varying lighting conditions than its Euclidean counterpart. The proposed matrix format for multichannel images enhances classification accuracy while reducing the number of parameters. Regarding segmentation, our method shows promising results, provided with a small class-imbalanced training data set. Proposed methodology achieves higher accuracy than prior work benchmarks and a small-scale CNN while maintaining a significantly lower parameter count. Notably, it is interpretable and accurate in scenarios where limited and unbalanced training data are available.

Artificial intelligence for nonmelanoma skin cancer

Nonmelanoma skin cancers (NMSCs) are among the top five most common cancers globally. NMSC is an area with great potential for novel application of diagnostic tools including artificial intelligence (AI). In this scoping review, we aimed to describe the applications of AI in the diagnosis and treatment of NMSC. Twenty-nine publications described AI applications to dermatopathology including lesion classification and margin assessment. Twenty-five publications discussed AI use in clinical image analysis, showing that algorithms are not superior to dermatologists and may rely on unbalanced, nonrepresentative, and nontransparent training data sets. Sixteen publications described the use of AI in cutaneous surgery for NMSC including use in margin assessment during excisions and Mohs surgery, as well as predicting procedural complexity. Eleven publications discussed spectroscopy, confocal microscopy, thermography, and the AI algorithms that analyze and interpret their data. Ten publications pertained to AI applications for the discovery and use of NMSC biomarkers. Eight publications discussed the use of smartphones and AI, specifically how they enable clinicians and patients to have increased access to instant dermatologic assessments but with varying accuracies. Five publications discussed large language models and NMSC, including how they may facilitate or hinder patient education and medical decision-making. Three publications pertaining to the skin of color and AI for NMSC discussed concerns regarding limited diverse data sets for the training of convolutional neural networks. AI demonstrates tremendous potential to improve diagnosis, patient and clinician education, and management of NMSC. Despite excitement regarding AI, data sets are often not transparently reported, may include low-quality images, and may not include diverse skin types, limiting generalizability. AI may serve as a tool to increase access to dermatology services for patients in rural areas and save health care dollars. These benefits can only be achieved, however, with consideration of potential ethical costs.

Generating Images of Rare Concepts Using Pre-trained Diffusion Models

Text-to-image diffusion models can synthesize high quality images, but they have various limitations. Here we highlight a common failure mode of these models, namely, generating uncommon concepts and structured concepts like hand palms. We show that their limitation is partly due to the long-tail nature of their training data: web-crawled data sets are strongly unbalanced, causing models to under-represent concepts from the tail of the distribution. We characterize the effect of unbalanced training data on text-to-image models and offer a remedy. We show that rare concepts can be correctly generated by carefully selecting suitable generation seeds in the noise space, using a small reference set of images, a technique that we call SeedSelect. SeedSelect does not require retraining or finetuning the diffusion model. We assess the faithfulness, quality and diversity of SeedSelect in creating rare objects and generating complex formations like hand images, and find it consistently achieves superior performance. We further show the advantage of SeedSelect in semantic data augmentation. Generating semantically appropriate images can successfully improve performance in few-shot recognition benchmarks, for classes from the head and from the tail of the training data of diffusion models.

FM-3DFR: Facial Manipulation-Based 3-D Face Reconstruction.

3-D Morphable model (3DMM) has widely benefited 3-D face-involved challenges given its parametric facial geometry and appearance representation. However, previous 3-D face reconstruction methods suffer from limited power in facial expression representation due to the unbalanced training data distribution and insufficient ground-truth 3-D shapes. In this article, we propose a novel framework to learn personalized shapes so that the reconstructed model well fits the corresponding face images. Specifically, we augment the dataset following several principles to balance the facial shape and expression distribution. A mesh editing method is presented as the expression synthesizer to generate more face images with various expressions. Besides, we improve the pose estimation accuracy by transferring the projection parameter into the Euler angles. Finally, a weighted sampling method is proposed to improve the robustness of the training process, where we define the offset between the base face model and the ground-truth face model as the sampling probability of each vertex. The experiments on several challenging benchmarks have demonstrated that our method achieves state-of-the-art performance.

Prospectivity modelling of critical mineral deposits using a generative adversarial network with oversampling and positive-unlabelled bagging

The demand for critical minerals is rapidly increasing worldwide, yet future global supply remains uncertain due to the difficulty in discovering new deposits using traditional methods. To increase the success rate of exploration projects for these vital resources, the use of artificial intelligence is continuously increasing for big and complex data analysis. This study proposes a new machine learning-based framework that tackles common problems associated with exploring critical mineral deposits, such as the shortage of known mineral occurrences, challenges in selecting negative samples in barren regions, and unbalanced training data. Our framework combines an improved generative adversarial network with positive and unlabelled learning to enhance efficiency. To test the performance of the framework, we create prospectivity maps of mafic–ultramafic intrusion-hosted mineralisation for cobalt, chromium, and nickel in the Gawler Craton, South Australia. The models are trained on a carefully selected set of independent features based on a conceptual model derived from open-access exploration data, resulting in high and stable performance. The prospectivity maps show a strong spatial correlation between high probabilities and known mineral occurrences and predict potential greenfield regions for future exploration. Our models demonstrate a significantly higher accuracy compared to a conventional approach using a standard random forest classifier and reveal that geophysical features play a crucial role in mapping prospective regions of critical minerals. Overall, our framework has the potential to significantly enhance critical mineral exploration by providing a more accurate and efficient approach to identifying prospective regions for future mining operations.

Random forest classification for volcanogenic massive sulfide mineralization in the Rouyn-Noranda Area, Quebec

Random forest (RF) classification was applied to 37 predictor maps (vectors to mineralization) producing a Mineral Prospectivity Map (MPM) for volcanogenic massive sulfide (VMS) mineralization in the Noranda District, Abitibi subprovince, which is host to ∼ 20 VMS deposits and numerous subeconomic occurrences. The predictor maps were created using geological, geochemical, and geophysical data, and the known VMS deposits were used to train the RF classifier. The RF model was applied on two regions of interest (ROI) to investigate the effect of different sized ROIs on the results. Five sets of balanced and unbalanced training data were used in the experiments to examine the sensitivity of RF to different sets of training data. The probability/prospectivity maps showed very high success rate of classification with regard to training data particularly for the training sets with higher ratio of non-deposits/deposits. Accuracy assessment of classification maps using cross-validation also showed very high accuracy when assessed by the training data in all experiments again giving a higher accuracy for the training sets with higher ratios of non-deposits/deposits, however the later showed very low accuracy when applying k-fold cross-validation or when assessed by 15 VMS showings used as the test data indicating an overfitting problem with the unbalanced data. In most of the experiments, principal component 4 (PC4) of geochemical analysis, proximity to synvolcanic tonalite-trondhjemite-granodiorite (TTG) intrusions, lithology, and sericite alteration maps showed higher predictive power. The importance of individual variables changed when the ROI was changed to a smaller area suggesting that RF is sensitive to the study area.

Machine learning clinical decision support for interdisciplinary multimodal chronic musculoskeletal pain treatment.

Chronic musculoskeletal pain is a prevalent condition impacting around 20% of people globally; resulting in patients living with pain, fatigue, restricted social and employment capacity, and reduced quality of life. Interdisciplinary multimodal pain treatment programs have been shown to provide positive outcomes by supporting patients modify their behavior and improve pain management through focusing attention on specific patient valued goals rather than fighting pain. Given the complex nature of chronic pain there is no single clinical measure to assess outcomes from multimodal pain programs. Using Centre for Integral Rehabilitation data from 2019-2021 (n = 2,364), we developed a multidimensional machine learning framework of 13 outcome measures across 5 clinically relevant domains including activity/disability, pain, fatigue, coping and quality of life. Machine learning models for each endpoint were separately trained using the most important 30 of 55 demographic and baseline variables based on minimum redundancy maximum relevance feature selection. Five-fold cross validation identified best performing algorithms which were rerun on deidentified source data to verify prognostic accuracy. Individual algorithm performance ranged from 0.49 to 0.65 AUC reflecting characteristic outcome variation across patients, and unbalanced training data with high positive proportions of up to 86% for some measures. As expected, no single outcome provided a reliable indicator, however the complete set of algorithms established a stratified prognostic patient profile. Patient level validation achieved consistent prognostic assessment of outcomes for 75.3% of the study group (n = 1,953). Clinician review of a sample of predicted negative patients (n = 81) independently confirmed algorithm accuracy and suggests the prognostic profile is potentially valuable for patient selection and goal setting. These results indicate that although no single algorithm was individually conclusive, the complete stratified profile consistently identified patient outcomes. Our predictive profile provides promising positive contribution for clinicians and patients to assist with personalized assessment and goal setting, program engagement and improved patient outcomes.

DWin-HRFormer: A High-Resolution Transformer Model With Directional Windows for Semantic Segmentation of Urban Construction Land

In this paper, a deep neural network for semantic segmentation of high-resolution remote sensing images is proposed for urban construction land classification. The network follows a high-resolution network architecture. Specifically, a directional self-attention on the paths of different resolutions is proposed, aiming to correct the directional bias caused by the attention of strip windows during the model learning, while also reducing the computational complexity, and allowing the model to improve both the accuracy and speed. At the end of the network, a distributed alignment module with spatial information is constructed to train additional learnable parameters, to adjust the biased decision boundaries through a two-stage learning strategy, and alleviate the problem of accuracy degradation due to the unbalanced training data. We tested the proposed method and compared it with the current state-of-the-art semantic segmentation methods on the Luojia-FGLC and WHDLD datasets, and the proposed one obtained the best performance. We also verified the effectiveness of each component of the network through ablation experiments. The code and model will be available at https://github.com/Zhzhyd/DWin-HRFormer.

LGBM: An Intrusion Detection Scheme for Resource-Constrained End Devices in Internet of Things

The intrusion detection schemes (IDSs) based on the Gradient Boosting Decision Tree (GBDT) face three problems: unbalanced training data distribution, large dimensionality of data features, and difficulty in model parameter optimization, which lead to weak monitoring capability and high false positive rate. For the problem of unbalanced training data distribution, we make the one-sided gradient oversampling algorithm to ensure the balance between the data of each category. To tackle the problem of the large dimensionality of data features, we develop a hierarchical cross-validation algorithm for binding mutually exclusive features. To address the problem of difficulty in model parameter optimization, we design a Bayesian optimization algorithm to make the model parameter search process more targeted and reduce the model training cost by establishing functional relationships between hyperparameters and target functions. The detailed experimental results show that the scheme can effectively solve the problems of data imbalance, high-dimensional data features, and low parameter finding efficiency, and improve the model’s ability to monitor the attack behavior.

Edge-to-Cloud IIoT for Condition Monitoring in Manufacturing Systems with Ubiquitous Smart Sensors.

The Industrial Internet of Things (IIoT) connects industrial assets to ubiquitous smart sensors and actuators to enhance manufacturing and industrial processes. Data-driven condition monitoring is an essential technology for intelligent manufacturing systems to identify anomalies from malfunctioning equipment, prevent unplanned downtime, and reduce the operation costs by predictive maintenance without interrupting normal machine operations. However, data-driven condition monitoring requires massive data collected from smart sensors to be transmitted to the cloud for further processing, thereby contributing to network congestion and affecting the network performance. Furthermore, unbalanced training data with very few labelled anomalies limit supervised learning models because of the lack of sufficient fault data for the training process in anomaly detection algorithms. To address these issues, we proposed an IIoT-based condition monitoring system with an edge-to-cloud architecture and computed the relative wavelet energy as feature vectors on the edge layer to reduce the network traffic overhead. We also proposed an unsupervised deep long short-term memory (LSTM) network module for anomaly detection. We implemented the proposed IIoT condition monitoring system for a manufacturing machine in a real shop site to evaluate our proposed solution. Our experimental results verify the effectiveness of our approach which can not only reduce the network traffic overhead for the IIoT but also detect anomalies accurately.

Exploring Unsupervised Learning with Clustering and Deep Autoencoder to Detect DDoS Attack

<p>With the proliferation of services available on the Internet, network attacks have become one of the seri-ous issues. The distributed denial of service (DDoS) attack is such a devastating attack, which poses an enormous threat to network communication and applications and easily disrupts services. To defense against DDoS attacks effectively, this paper proposes a novel DDoS attack detection method that trains detection models in an unsupervised learning manner using preprocessed and unlabeled normal network traffic data, which can not only avoid the impact of unbalanced training data on the detection model per-formance but also detect unknown attacks. Specifically, the proposed method firstly uses Balanced Itera-tive Reducing and Clustering Using Hierarchies algorithm (BIRCH) to pre-cluster the normal network traf-fic data, and then explores autoencoder (AE) to build the detection model in an unsupervised manner based on the cluster subsets. In order to verify the performance of our method, we perform experiments on benchmark network intrusion detection datasets KDDCUP99 and UNSWNB15. The results show that, compared with the state-of-the-art DDoS detection models that used supervised learning and unsuper-vised learning, our proposed method achieves better performance in terms of detection accuracy rate and false positive rate (FPR).</p> <p> </p>

SASMOTE: A Self-Attention Oversampling Method for Imbalanced CSI Fingerprints in Indoor Positioning Systems.

WiFi localization based on channel state information (CSI) fingerprints has become the mainstream method for indoor positioning due to the widespread deployment of WiFi networks, in which fingerprint database building is critical. However, issues, such as insufficient samples or missing data in the collection fingerprint database, result in unbalanced training data for the localization system during the construction of the CSI fingerprint database. To address the above issue, we propose a deep learning-based oversampling method, called Self-Attention Synthetic Minority Oversampling Technique (SASMOTE), for complementing the fingerprint database to improve localization accuracy. Specifically, a novel self-attention encoder-decoder is firstly designed to compress the original data dimensionality and extract rich features. The synthetic minority oversampling technique (SMOTE) is adopted to oversample minority class data to achieve data balance. In addition, we also construct the corresponding CSI fingerprinting dataset to train the model. Finally, extensive experiments are performed on different data to verify the performance of the proposed method. The results show that our SASMOTE method can effectively solve the data imbalance problem. Meanwhile, the improved location model, 1D-MobileNet, is tested on the balanced fingerprint database to further verify the excellent performance of our proposed methods.

Progressive Training Technique with Weak-Label Boosting for Fine-Grained Classification on Unbalanced Training Data

In practical classification tasks, the sample distribution of the dataset is often unbalanced; for example, this is the case in a dataset that contains a massive quantity of samples with weak labels and for which concrete identification is unavailable. Even in samples with exact labels, the number of samples corresponding to many labels is small, resulting in difficulties in learning the concepts through a small number of labeled samples. In addition, there is always a small interclass variance and a large intraclass variance among categories. Weak labels, few-shot problems, and fine-grained analysis are the key challenges affecting the performance of the classification model. In this paper, we develop a progressive training technique to address the few-shot challenge, along with a weak-label boosting method, by considering all of the weak IDs as negative samples of every predefined ID in order to take full advantage of the more numerous weak-label data. We introduce an instance-aware hard ID mining strategy in the classification loss and then further develop the global and local feature-mapping loss to expand the decision margin. We entered the proposed method into the Kaggle competition, which aims to build an algorithm to identify individual humpback whales in images. With a few other common training tricks, the proposed approach won first place in the competition. All three problems (weak labels, few-shot problems, and fine-grained analysis) exist in the dataset used in the competition. Additionally, we applied our method to CUB-2011 and Cars-196, which are the most widely-used datasets for fine-grained visual categorization tasks, and achieved respective accuracies of 90.1% and 94.9%. This experiment shows that the proposed method achieves the optimal effect compared with other common baselines, and verifies the effectiveness of our method. Our solution has been made available as an open source project.

Deep-Forest-Based Encrypted Malicious Traffic Detection

The SSL/TLS protocol is widely used in data encryption transmission. Aiming at the problem of detecting SSL/TLS-encrypted malicious traffic with small-scale and unbalanced training data, a deep-forest-based detection method called DF-IDS is proposed in this paper. According to the characteristics of SSL/TSL protocol, the network traffic was split into sessions according to the 5-tuple information. Each session was then transformed into a two-dimensional traffic image as the input of a deep-learning classifier. In order to avoid information loss and improve the detection efficiency, the multi-grained cascade forest (gcForest) framework was simplified with only cascade structure, which was named cascade forest (CaForest). By integrating random forest and extra trees in the CaForest framework, an end-to-end high-precision detector for small-scale and unbalanced SSL/TSL encrypted malicious traffic was realized. Compared with other deep-learning-based methods, the experimental results showed that the detection rate of DF-IDS was 6.87% to 29.5% higher than that of other methods on a small-scale and unbalanced dataset. The advantage of DF-IDS was more obvious in the multi-classification case.

Remaining useful life prediction of bearing based on stacked autoencoder and recurrent neural network

Remaining Useful Life (RUL) prediction play a crucial part in bearing maintenance, which directly affects the production efficiency and safety of equipment. Moreover, the accuracy of the prediction model is constrained by the feature extraction process and full life data of bearings. In this paper, the life prediction method of faulty rolling bearing with limited data is presented including degradation state model and RUL prediction model. In order to obtain health indication (HI) without human interference in the degradation state modeling stage, the bottleneck structure of Stacked Autoencoder (SAE) is utilized to fuse the four selected features into one HI using Intelligent Maintenance Systems (IMS) bearing dataset as training sample. In RUL prediction model, the Long Short-Term Memory (LSTM) neural network is carried out to establish the model with Standard deviation (Std) input and HI training label. In order to solve the problem of large training error caused by insufficient data in the failure stage of bearing acceleration test, the third-order spline curve interpolation is utilized to enhance the data points. Through parameter analysis, the RMSE and MAE of the test set on the prediction model are 0.032582 and 0.024038, respectively. Furthermore, the effectiveness of the proposed method is further validated by dataset from Case Western Reserve University (CWRU) with different bearing fault degrees. The analysis indicates that the RUL prediction of bearing fault data is consistent with the size of artificial added faults, that is,the more severe the fault the shorter the time of remaining life. The results validate that the proposed method can effectively extract the bearing health state by incorporating feature fusion and establish accurately prediction model for bearing remaining life.

Research on Algorithm of DRC Catalog Generation Based on Machine Learning

Aiming at the current logic and expansion issues in data governance caused by data scheduling in two directions: business retrieval and data processing. Data Oriented Architecture proposes a Registration Intermediate Library(RIL) and a Catalog Intermediate Library(CIL) to achieve separation of management and application. Among them, the business-oriented catalog intermediate library automatically classifies and labels data registration information through machine learning, and realizes the automatic generation of catalogs.There are many text classification methods based on machine learning, but the model obtained through unbalanced data set training often has performance degradation. In order to solve this problem, category weights are introduced in the feature vector to reduce the influence of most samples on the model parameters. Experiments on THUCNews text classification data set show that the proposed method can effectively improve the performance of baseline system, solve the problem of unbalanced training data categories and solve the problem of automatic generation of DRC directory.

Novel MOA Fault Detection Technology Based on Small Sample Infrared Image

This paper proposes a novel metal oxide arrester (MOA) fault detection technology based on a small sample infrared image. The research is carried out from the detection process and data enhancement. A lightweight MOA identification and location algorithm is designed at the edge, which can not only reduce the amount of data uploaded, but also reduce the search space of cloud algorithm. In order to improve the accuracy and generalization ability of the defect detection model under the condition of small samples, a multi-model fusion detection algorithm is proposed. Different features of the image are extracted by multiple convolutional neural networks, and then multiple classifiers are trained. Finally, the weighted voting strategy is used for fault diagnosis. In addition, the extended model of fault samples is constructed by transfer learning and deep convolutional generative adversarial networks (DCGAN) to solve the problem of unbalanced training data sets. The experimental results show that the proposed method can realize the accurate location of arrester under the condition of small samples, and after the data expansion, the recognition rate of arrester anomalies can be improved from 83% to 85%, showing high effectiveness and reliability.

RF-GANs: A Method to Synthesize Retinal Fundus Images Based on Generative Adversarial Network.

Diabetic retinopathy (DR) is a diabetic complication affecting the eyes, which is the main cause of blindness in young and middle-aged people. In order to speed up the diagnosis of DR, a mass of deep learning methods have been used for the detection of this disease, but they failed to attain excellent results due to unbalanced training data, i.e., the lack of DR fundus images. To address the problem of data imbalance, this paper proposes a method dubbed retinal fundus images generative adversarial networks (RF-GANs), which is based on generative adversarial network, to synthesize retinal fundus images. RF-GANs is composed of two generation models, RF-GAN1 and RF-GAN2. Firstly, RF-GAN1 is employed to translate retinal fundus images from source domain (the domain of semantic segmentation datasets) to target domain (the domain of EyePACS dataset connected to Kaggle (EyePACS)). Then, we train the semantic segmentation models with the translated images, and employ the trained models to extract the structural and lesion masks (hereafter, we refer to it as Masks) of EyePACS. Finally, we employ RF-GAN2 to synthesize retinal fundus images using the Masks and DR grading labels. This paper verifies the effectiveness of the method: RF-GAN1 can narrow down the domain gap between different datasets to improve the performance of the segmentation models. RF-GAN2 can synthesize realistic retinal fundus images. Adopting the synthesized images for data augmentation, the accuracy and quadratic weighted kappa of the state-of-the-art DR grading model on the testing set of EyePACS increase by 1.53% and 1.70%, respectively.

LightFed: An Efficient and Secure Federated Edge Learning System on Model Splitting

With the integration of Artificial Intelligence (AI) and Internet of Things (IoT), the Federated Edge Learning (FEL), a promising computing framework is developing. However, there are still unsolved issues on communication efficiency and data security due to the huge models and unreliable transmission links. To address these issues, this paper proposes a novel federated edge learning system, called LightFed, where the edge nodes upload only vital partial local models, and successfully achieve lightweight communication and model aggregation. First, a novel model aggregation method Model Splitting and Splicing (MSS) and a Selective Parameter Transmission (SPT) scheme are proposed. By detecting the updating gradients of local parameters and filtering significant parameters, selective rotated transmission and efficient aggregation of local models are achieved. Second, a Training Filling Model (TFM) is proposed to infer the total data distribution of edge nodes, and train a filling model to mitigate the unbalanced training data without violating the data privacy of individual users. Moreover, a blockchain-powered confusion transmission mechanism is proposed for defending the attacks from external adversaries and protecting the model information. Finally, extensive experimental results demonstrate that our LightFed significantly outperforms the existing FEL systems in terms of communication efficiency and privacy security.