Cost-sensitive Learning Approach Research Articles

Machine learning enables automated screening for systematic reviews and meta-analysis in urology

PurposeTo investigate and implement semiautomated screening for meta-analyses (MA) in urology under consideration of class imbalance.MethodsMachine learning algorithms were trained on data from three MA with detailed information of the screening process. Different methods to account for class imbalance (Sampling (up- and downsampling, weighting and cost-sensitive learning), thresholding) were implemented in different machine learning (ML) algorithms (Random Forest, Logistic Regression with Elastic Net Regularization, Support Vector Machines). Models were optimized for sensitivity. Besides metrics such as specificity, receiver operating curves, total missed studies, and work saved over sampling were calculated.ResultsDuring training, models trained after downsampling achieved the best results consistently among all algorithms. Computing time ranged between 251 and 5834 s. However, when evaluated on the final test data set, the weighting approach performed best. In addition, thresholding helped to improve results as compared to the standard of 0.5. However, due to heterogeneity of results no clear recommendation can be made for a universal sample size. Misses of relevant studies were 0 for the optimized models except for one review.ConclusionIt will be necessary to design a holistic methodology that implements the presented methods in a practical manner, but also takes into account other algorithms and the most sophisticated methods for text preprocessing. In addition, the different methods of a cost-sensitive learning approach can be the subject of further investigations.

Interpretable machine learning models for failure cause prediction in imbalanced oil pipeline data

Pipelines are critical arteries in the oil and gas industry and require massive capital investment to safely construct networks that transport hydrocarbons across diverse environments. However, these pipeline systems are prone to integrity failure, which results in significant economic losses and environmental damage. Accurate prediction of pipeline failure events using historical oil pipeline accident data enables asset managers to plan sufficient maintenance, rehabilitation, and repair activities to prevent catastrophic failures. However, learning the complex interdependencies between pipeline attributes and rare failure events presents several analytical challenges. This study proposes a novel machine learning (ML) framework to accurately predict pipeline failure causes on highly class-imbalanced data compiled by the United States Pipeline and Hazardous Materials Safety Administration. Natural language processing techniques were leveraged to extract informative features from unstructured text data. Furthermore, class imbalance in the dataset was addressed via oversampling and intrinsic cost-sensitive learning (CSL) strategies adapted for the multi-class case. Nine machine and deep learning architectures were benchmarked, with LightGBM demonstrating superior performance. The integration of CSL yielded an 86% F1 score and a 0.82 Cohen kappa score, significantly advancing prior research. This study leveraged a comprehensive Shapley Additive explanation analysis to interpret the predictions from the LightGBM algorithm, revealing the key factors driving failure probabilities. Leveraging sentiment analysis allowed the models to capture a richer, more multifaceted representation of the textual data. This study developed a novel CSL approach that integrates domain knowledge regarding the varying cost impacts of misclassifying different failure types into ML models. This research demonstrated an effective fusion of text insights from inspection reports with structured pipeline data that enhances model interpretability. The resulting AI modeling framework generated data-driven predictions of the causes of failure that could enable transportation agencies with actionable insights. These insights enable tailored preventative maintenance decisions to proactively mitigate emerging pipeline failures.

Cost-sensitive learning for imbalanced medical data: a review

Integrating Machine Learning (ML) in medicine has unlocked many opportunities to harness complex medical data, enhancing patient outcomes and advancing the field. However, the inherent imbalanced distribution of medical data poses a significant challenge, resulting in biased ML models that perform poorly on minority classes. Mitigating the impact of class imbalance has prompted researchers to explore various strategies, wherein Cost-Sensitive Learning (CSL) arises as a promising approach to improve the accuracy and reliability of ML models. This paper presents the first review of CSL for imbalanced medical data. A comprehensive exploration of the existing literature encompassed papers published from January 2010 to December 2022 and sourced from five major digital libraries. A total of 173 papers were selected, analysed, and classified based on key criteria, including publication years, channels and sources, research types, empirical types, medical sub-fields, medical tasks, CSL approaches, strengths and weaknesses of CSL, frequently used datasets and data types, evaluation metrics, and development tools. The results indicate a noteworthy publication rise, particularly since 2020, and a strong preference for CSL direct approaches. Data type analysis unveiled diverse modalities, with medical images prevailing. The underutilisation of cost-related metrics and the prevalence of Python as the primary programming tool are highlighted. The strengths and weaknesses analysis covered three aspects: CSL strategy, CSL approaches, and relevant works. This study serves as a valuable resource for researchers seeking to explore the current state of research, identify strengths and gaps in the existing literature and advance CSL’s application for imbalanced medical data.

Examining different cost ratio frameworks for decision rule machine learning algorithms in diagnostic application.

Artificial Intelligence (AI) plays a pivotal role in the diagnosis of health conditions ranging from general well-being to critical health issues. In the realm of health diagnostics, an often overlooked but critical aspect is the consideration of cost-sensitive learning, a facet that this study prioritizes over the non-invasive nature of the diagnostic process whereas the other standard metrics such as accuracy and sensitivity reflect weakness in error profile. This research aims to investigate the total cost of misclassification (Total Cost) by decision rule Machine Learning (ML) algorithms implemented in Java platforms such as DecisionTable, JRip, OneR, and PART. An augmented dataset with conjunctiva images along candidates' demographic and anthropometric features under supervised learning is considered with a specific emphasis on cost-sensitive classification. The opted decision rule classifiers use the text features, additionally the image feature 'a* value of CIELAB color space' extracted from the conjunctiva digital images as input attributes. The pre-processing consists of amalgamating text and image features on a uniform scale, normalizing. Then the 10-fold cross-validation enables the classification of samples into two categories: the presence or absence of the anemia. This study utilizes the Cost Ratio (ρ) extracted from the cost matrix to meticulously monitor the Total Cost in four different cost ratio methodologies namely Uniform (U), Uniform Inverted (UI), Non-Uniform (NU), and Non-Uniform Inverted (NUI). It has been established that the PART classifier stands out as the top performer in this binary classification task, yielding the lowest mean total cost of 629.9 compared to other selected classifiers. Moreover, it demonstrates a comparatively lower standard deviation 335.9, and lower total cost across all four different cost ratio methodologies. The ranking of algorithm performance goes as follows: PART, JRIP, DecisionTable, and OneR. The significance of adopting a cost-sensitive learning approach is emphasized showing the PART classifier's consistent performance within the proposed framework for learning the anemia dataset. This emphasis on cost-sensitive learning not only enhances the recommendations in diagnosis but also holds the potential for substantial cost savings and makes it a noteworthy focal point in the advancement of AI-driven health care.

Detection of Hate Speech and Offensive Language CodeMix Text in Dravidian Languages Using Cost-Sensitive Learning Approach

Detection of Hate Speech and Offensive Language CodeMix Text in Dravidian Languages Using Cost-Sensitive Learning Approach

An adaptive cost-sensitive learning approach in neural networks to minimize local training–test class distributions mismatch

We design an adaptive learning algorithm for binary classification problems whose objective is to reduce the cost of misclassified instances derived from the consequences of errors. Our algorithm (Adaptive Cost-Sensitive Learning — AdaCSL) adaptively adjusts the loss function to bridge the difference between the class distributions between subgroups of samples in the training and validation data sets. This adjustment is made for samples with similar predicted probabilities, in such a way that the local cost decreases. This process usually leads to a reduction in cost when applied to the test data set (i.e., local training–test class distributions mismatch). We present empirical evidence that neural networks used with the proposed algorithm yield better cost results on several data sets compared to other approaches. In addition, the proposed AdaCSL algorithm can optimize evaluation metrics other than cost. We present an experiment that demonstrates how utilizing the AdaCSL algorithm generates superior accuracy results. The AdaCSL algorithm can be used for applications in which the training set is noisy or when large variability may occur between the training and validation data sets, such as the classification of disease severity for a given subject based on other subjects. Our code is available at https://github.com/OhadVolk/AdaCSL.

Handling imbalanced class in melanoma: Kemeny–Young rule based optimal rank aggregation and Self-Adaptive Differential Evolution Optimization

Melanoma, despite its relatively low incidence compared to other types of skin cancer, accounts for a significant proportion of skin cancer-related deaths. Early detection of melanoma is crucial for improving patient survival rates. Deep learning algorithms, which heavily rely on data, are widely used in melanoma detection. However, the performance of these algorithms is greatly influenced by the distribution of the dataset, particularly class imbalance. In this manuscript, the authors present a novel method based on the Kemeny–Young rule for optimal rank aggregation to address the class imbalance problem in melanoma detection. The proposed approach aims to reduce class bias and enhance overall classification accuracy. Furthermore, a cost-sensitive learning approach is introduced to improve the classifier’s ability to handle class imbalance effectively. This novel cost-sensitive learning method utilizes Self-Adaptive Differential Evolution Optimization to determine optimal weights for each class. Our approach differs from traditional methods that assign weights based on predefined criteria. To evaluate the effectiveness of the proposed methods, extensive experiments and ablation studies are conducted on the highly imbalanced ISIC 2020 dataset, which is widely used in melanoma detection research. The Kemeny–Young rule-based majority voting achieves an overall error rate of 2.44%, while the cost-sensitive learning based on the Self-Adaptive Differential Evolution approach achieves an even lower error rate of 1.99%. Moreover, the proposed method achieves a sensitivity of 87.93% and a specificity of 98.19%. These experimental results demonstrate the competitiveness and effectiveness of the proposed methods in addressing the challenges posed by class imbalance and improving the accuracy of melanoma detection. By effectively mitigating class imbalance, these methods improve the accuracy and reliability of melanoma detection, thus offering valuable insights for developing advanced computer-aided diagnosis systems in dermatology. The relevant codes for our proposed approach are publicly available at: https://github.com/ctrl-gaurav/Handling-Imbalanced-Class-in-Melanoma.

Deep Learning-Based Network Intrusion Detection System for Internet of Medical Things

This article presents a deep learning-based approach for network-based intrusion detection in the Internet of medical things (IoMT) systems using features of network flows and patient biometrics. The proposed approach effectively learns optimal feature representation by passing the information of network flows and patient biometrics into more than one hidden layer of deep learning. The network includes a global attention layer which helps to effectively extract the optimal features from the spatial and temporal features of deep learning. To avoid data imbalance, a cost-sensitive learning approach is integrated into the deep learning model. The proposed model showed a 10-fold cross-validation accuracy of 95 percent on network features, 89 percent on patient biometrics, and 99 percent on combined features. In addition to the IoMT environment, the robustness and generalization ability of the proposed model is shown by conducting experiments on other network-based intrusion datasets. The proposed approach outperformed the existing methods in all the test cases mainly showing a 3.9 percent higher accuracy on the IoMT intrusion dataset. The proposed model can be used as an IoMT network monitoring tool to safeguard the IoMT devices and networks from attackers inside the healthcare and medical environment.

RUE: A robust personalized cost assignment strategy for class imbalance cost-sensitive learning

Cost-sensitive learning is a popular paradigm to address class-imbalance learning (CIL) problem. Traditional cost-sensitive learning approaches always solve CIL problem by assigning a constant higher training error penalty for all minority instances than that of majority instances, but ignore the significance of location information. Therefore, several recent studies began to focus on the personalized cost assignment, i.e., designating different costs for different instances based on their location information. The emerging personalized cost-sensitive approaches always perform better than those traditional ones; however, the estimation for location information may be inaccurate as it is apt to be impacted by data density variation. To address this problem, we propose a novel location information estimation and cost assignment strategy called RUE. Unlike previous approaches, our proposed strategy explores location information by an indirect way: the error rate feed backed from a random undersampling ensemble. The strategy is robust towards data distribution, and is helpful for accurately estimating the significance of each instance regardless the complexity of data distribution. In context of Fuzzy Support Vector Machine (FSVM) and Weighted Extreme Learning Machine (WELM), the proposed cost assignment strategy is compared with several popular and state-of-the-art approaches, and the results show its effectiveness and superiority.

A multi‐attention dynamic graph convolution network with cost‐sensitive learning approach to road‐level and minute‐level traffic accident prediction

Traffic accident prediction on road levels and minute levels plays an important role in optimizing public safety and improving traffic infrastructure. However, there are still some challenges in this work. Firstly, the dynamic factors (e.g. traffic flow) affecting traffic accidents make the road network have dynamic spatio-temporal dependency, which leads to biased prediction results. Secondly, the occurrence of traffic accidents is a small probability event, which brings about zero-inflation problem. To address aforementioned problems, the authors propose a Multi-Attention Dynamic Graph Convolution Network with Cost Sensitive Learning approach (MADGCN). Specifically, in the spatial dimension, MADGCN calculates the attention scores of different types of dynamic factors through attention mechanism to simulate the different influence degrees of different factors, and models dynamic inter-road spatial correlation through Graph Convolution Network (GCN). In the temporal aspect, MADGCN adaptively models the dynamic temporal correlations through self-attention blocks. In addition, MADGCN improves the loss function based on cost-sensitive learning strategy to increase the cost of false classification of positive samples, so as to accurately mine sparse positive samples. Experimental results on two real-world traffic accident datasets demonstrate the superiority of MADGCN. Compared with the existing state-of-the-art methods, F1-score of MADGCN on two real-world traffic accident datasets is improved by 11.61% and 9.15%, respectively.

A hybrid cost-sensitive machine learning approach for the classification of intelligent disease diagnosis

Automatic risk classification of diseases is one of the most significant health problems in medical and healthcare domain. However, the related studies are relative scarce. In this paper, we design an intelligent diagnosis model based on optimal machine learning algorithms with rich clinical data. First, the disease risk classification problem based on machine learning is defined. Then, the K-means clustering algorithm is used to validate the class label of given data, thereby removing misclassified instances from the original dataset. Furthermore, naive Bayesian algorithm is applied to build the final classifier by using 10-fold cross-validation method. In addition, a novel class-specific attribute weighted approach is adopted to alleviate the conditional independence assumption of naive Bayes, which means we assign each disease attribute a specific weight for each class. Last but not least, a hybrid cost-sensitive disease risk classification model is formulated, and a practical example from the University of California Irvine (UCI) machine learning database is used to illustrate the potential of the proposed method. Experimental results demonstrate that the approach is competitive with the state-of-the-art classifiers.

Intelligent Fault Diagnosis of Industrial Robot Based on Multiclass Mahalanobis-Taguchi System for Imbalanced Data.

One of the biggest challenges for the fault diagnosis research of industrial robots is that the normal data is far more than the fault data; that is, the data is imbalanced. The traditional diagnosis approaches of industrial robots are more biased toward the majority categories, which makes the diagnosis accuracy of the minority categories decrease. To solve the imbalanced problem, the traditional algorithm is improved by using cost-sensitive learning, single-class learning and other approaches. However, these algorithms also have a series of problems. For instance, it is difficult to estimate the true misclassification cost, overfitting, and long computation time. Therefore, a fault diagnosis approach for industrial robots, based on the Multiclass Mahalanobis-Taguchi system (MMTS), is proposed in this article. It can be classified the categories by measuring the deviation degree from the sample to the reference space, which is more suitable for classifying imbalanced data. The accuracy, G-mean and F-measure are used to verify the effectiveness of the proposed approach on an industrial robot platform. The experimental results show that the proposed approach’s accuracy, F-measure and G-mean improves by an average of 20.74%, 12.85% and 21.68%, compared with the other five traditional approaches when the imbalance ratio is 9. With the increase in the imbalance ratio, the proposed approach has better stability than the traditional algorithms.

An Improvised Random Forest Model for Breast Cancer Classification

Breast Cancer is considered as the most common cancer in females with high incidence rate. The evolution of modern facilities has helped in reducing the mortality rate, yet the incidence is still the highest among all cancers affecting women. Early diagnosis is a predominant factor for survival. Hence techniques to assist the current modalities are essential. Machine learning techniques have been used so as to produce better prediction and classification models which will aid in better and earlier disease diagnosis and classification. Random Forest is a supervised machine learning classifier that helps in better classification. Random Forests are applied to the Wisconsin breast cancer dataset and the performance of the classifier is evaluated for breast cancer classification. Here in this study an improvised random forest model which uses a cost sensitive learning approach for classification is proposed and it is found to have a better performance than the traditional random forest approach. The model gave an accuracy of 97.51%.

A Cost-Sensitive Deep Learning-Based Approach for Network Traffic Classification

Network traffic classification (NTC) plays an important role in cyber security and network performance, for example in intrusion detection and facilitating a higher quality of service. However, due to the unbalanced nature of traffic datasets, NTC can be extremely challenging and poor management can degrade classification performance. While existing NTC methods seek to re-balance data distribution through resampling strategies, such approaches are known to suffer from information loss, overfitting, and increased model complexity. To address these challenges, we propose a new cost-sensitive deep learning approach to increase the robustness of deep learning classifiers against the imbalanced class problem in NTC. First, the dataset is divided into different partitions, and a cost matrix is created for each partition by considering the data distribution. Then, the costs are applied to the cost function layer to penalize classification errors. In our approach, costs are diverse in each type of misclassification because the cost matrix is specifically generated for each partition. To determine its utility, we implement the proposed cost-sensitive learning method in two deep learning classifiers, namely: stacked autoencoder and convolution neural networks. Our experiments on the ISCX VPN-nonVPN dataset show that the proposed approach can obtain higher classification performance on low-frequency classes, in comparison to three other NTC methods.

A dynamic time warping approach for handling class imbalanced medical datasets with missing values: A case study of protein localization site prediction

Class imbalanced medical datasets, such as cancer prediction, contain imbalanced numbers of data in different classes leading to skewed class distribution, which makes it very difficult for a classifier to distinguish between minority (i.e. cancer) and majority (i.e. non-cancer) classes. Related studies in the literature have proposed different types of solutions for the class imbalance problem including data level, algorithmic level, and cost-sensitive learning approaches. However, none of these potential solutions have considered the issue of missing attribute values residing in the class imbalanced medical datasets, especially for the minority class. Missing value imputation is commonly used for the construction of some models where statistical or machine learning techniques are used to produce estimations to replace the missing values. However, the existing imputation methods require a certain number of observed data to produce their estimations, the major challenge for them being that the amount of observed data (with no missing values) in the minority class is very limited, or that some data are not complete. In this paper, we proposed a novel approach, namely Dynamic Time Warping-based Imputation (DTWI), to handle class imbalanced datasets with missing values. Based on the similarity measurement technique of DTW, all of the data (with or without missing values) in the minority class can be used for missing value imputation. The experimental results based on 10 different class imbalanced medical datasets show that when the missing rates in the minority classes are smaller than 30%, DTWI performs similarly to the baseline K-NN imputation method and better than the mean/mode imputation and case deletion methods. When the missing rates are larger than 30%, DTWI significantly outperform the other techniques.

Cost-sensitive meta-learning framework

PurposeThis paper aims to describe the use of a meta-learning framework for recommending cost-sensitive classification methods with the aim of answering an important question that arises in machine learning, namely, “Among all the available classification algorithms, and in considering a specific type of data and cost, which is the best algorithm for my problem?”Design/methodology/approachThis paper describes the use of a meta-learning framework for recommending cost-sensitive classification methods for the aim of answering an important question that arises in machine learning, namely, “Among all the available classification algorithms, and in considering a specific type of data and cost, which is the best algorithm for my problem?” The framework is based on the idea of applying machine learning techniques to discover knowledge about the performance of different machine learning algorithms. It includes components that repeatedly apply different classification methods on data sets and measures their performance. The characteristics of the data sets, combined with the algorithms and the performance provide the training examples. A decision tree algorithm is applied to the training examples to induce the knowledge, which can then be used to recommend algorithms for new data sets. The paper makes a contribution to both meta-learning and cost-sensitive machine learning approaches. Those both fields are not new, however, building a recommender that recommends the optimal case-sensitive approach for a given data problem is the contribution. The proposed solution is implemented in WEKA and evaluated by applying it on different data sets and comparing the results with existing studies available in the literature. The results show that a developed meta-learning solution produces better results than METAL, a well-known meta-learning system. The developed solution takes the misclassification cost into consideration during the learning process, which is not available in the compared project.FindingsThe proposed solution is implemented in WEKA and evaluated by applying it to different data sets and comparing the results with existing studies available in the literature. The results show that a developed meta-learning solution produces better results than METAL, a well-known meta-learning system.Originality/valueThe paper presents a major piece of new information in writing for the first time. Meta-learning work has been done before but this paper presents a new meta-learning framework that is costs sensitive.

Density-based weighting for imbalanced regression

In many real world settings, imbalanced data impedes model performance of learning algorithms, like neural networks, mostly for rare cases. This is especially problematic for tasks focusing on these rare occurrences. For example, when estimating precipitation, extreme rainfall events are scarce but important considering their potential consequences. While there are numerous well studied solutions for classification settings, most of them cannot be applied to regression easily. Of the few solutions for regression tasks, barely any have explored cost-sensitive learning which is known to have advantages compared to sampling-based methods in classification tasks. In this work, we propose a sample weighting approach for imbalanced regression datasets called DenseWeight and a cost-sensitive learning approach for neural network regression with imbalanced data called DenseLoss based on our weighting scheme. DenseWeight weights data points according to their target value rarities through kernel density estimation (KDE). DenseLoss adjusts each data point’s influence on the loss according to DenseWeight, giving rare data points more influence on model training compared to common data points. We show on multiple differently distributed datasets that DenseLoss significantly improves model performance for rare data points through its density-based weighting scheme. Additionally, we compare DenseLoss to the state-of-the-art method SMOGN, finding that our method mostly yields better performance. Our approach provides more control over model training as it enables us to actively decide on the trade-off between focusing on common or rare cases through a single hyperparameter, allowing the training of better models for rare data points.

A Comprehensive Analysis of Handling Imbalanced Dataset

Classification is a major obstacle in Machine Learning generally and also specific when tackling class imbalance problem. A dataset is said to be imbalanced if a class we are interested in falls to the minority class and appears scanty when compared to the majority class, the minority class is also known as the positive class while the majority class is also known as the negative class. Class imbalance has been a major bottleneck for Machine Learning scientist as it often leads to using wrong model for different purposes, this Survey will lead researchers to choose the right model and the best strategies to handle imbalance dataset in the course of tackling machine learning problems. Proper handling of class imbalance dataset could leads to accurate and good result. Handling class imbalance data in a conventional manner, especially when the level of imbalance is high may leads to accuracy paradox (an assumption of realizing 99% accuracy during evaluation process when the class distribution is highly imbalanced), hence imbalance class distribution requires special consideration, and for this purpose we dealt extensively on handling and solving imbalanced class problem in machine learning, such as Data Sampling Approach, Cost sensitive learning approach and Ensemble Approach.

Multiset Feature Learning for Highly Imbalanced Data Classification.

With the expansion of data, increasing imbalanced data has emerged. When the imbalance ratio (IR) of data is high, most existing imbalanced learning methods decline seriously in classification performance. In this paper, we systematically investigate the highly imbalanced data classification problem, and propose an uncorrelated cost-sensitive multiset learning (UCML) approach for it. Specifically, UCML first constructs multiple balanced subsets through random partition, and then employs the multiset feature learning (MFL) to learn discriminant features from the constructed multiset. To enhance the usability of each subset and deal with the non-linearity issue existed in each subset, we further propose a deep metric based UCML (DM-UCML) approach. DM-UCML introduces the generative adversarial network technique into the multiset constructing process, such that each subset can own similar distribution with the original dataset. To cope with the non-linearity issue, DM-UCML integrates deep metric learning with MFL, such that more favorable performance can be achieved. In addition, DM-UCML designs a new discriminant term to enhance the discriminability of learned metrics. Experiments on eight traditional highly class-imbalanced datasets and two large-scale datasets indicate that: the proposed approaches outperform state-of-the-art highly imbalanced learning methods and are more robust to high IR.

Cost-sensitive multiple-instance learning method with dynamic transactional data for personal credit scoring

We study how to assess an applicant’s credit risk with dynamic transactional data. The problem arises when an applicant applies for loans from financial institutions. A traditional credit-risk assessment model utilizes individual demographic and loan information from an application form. Nevertheless, dynamic transactional data is good indicators of an applicant’s credit risk. However, the lack of available data and the preexisting limitations of conventional approaches limit the use of the dynamic transactional data. In this study, we propose a cost-sensitive multiple-instance learning (MIL) approach to evaluate applicants’ credit scores that incorporate their dynamic transactional data and static individual information. Traditionally, MIL approaches can handle the variable number of input instances. However, to facilitate the implementation of MIL into credit scoring, we extend the MIL to consider the dynamic transactional data and cost-sensitive problem simultaneously. We compare our model with several benchmark MIL models by testing them on real-world data sets. Experimental results show that our model outperforms most benchmarks in many widely used criteria.