Misclassification Costs Research Articles

MiNB: Minority Sensitive Naïve Bayesian Algorithm for Multi-Class Classification of Unbalanced Data

The unbalanced nature of data makes it tough to achieve the desire performance goal for classification algorithms. The sub-optimal prediction system isn't a viable solution due to the high misclassification cost of minority events. Thus accurate imbalanced data classification could be a path changer for prediction in domains like medical diagnosis, judiciary, and disaster management systems. To date, most of the existing studies of imbalanced data are for the binary class dataset and supported by data sampling techniques that suffer from loss of information and over-fitting. In this paper, we present the modified naïve Bayesian algorithm for unbalanced data classification that eliminates the requirement of data level sampling. We compared our proposed model with the data sampling technique and cost-sensitive techniques. We use minority sensitive TP Rate, class-specific misclassification rate, and overall performance parameters such as accuracy, f-measure and G-mean. The result shows that our proposed algorithm shows a more optimal result for unbalanced data classification. Results shows reduction in misclassification rate and improve predictive performance for the minority class.

Public–Private Partnership Contract Failure Prediction Using Example-Dependent Cost-Sensitive Models

AbstractFailure of a public–private partnership (PPP) contract could cause heavy losses to sponsors. However, the current machine learning models neglect misclassification costs when predicting PPP...

Cost-sensitive learning strategies for high-dimensional and imbalanced data: a comparative study.

High dimensionality and class imbalance have been largely recognized as important issues in machine learning. A vast amount of literature has indeed investigated suitable approaches to address the multiple challenges that arise when dealing with high-dimensional feature spaces (where each problem instance is described by a large number of features). As well, several learning strategies have been devised to cope with the adverse effects of imbalanced class distributions, which may severely impact on the generalization ability of the induced models. Nevertheless, although both the issues have been largely studied for several years, they have mostly been addressed separately, and their combined effects are yet to be fully understood. Indeed, little research has been so far conducted to investigate which approaches might be best suited to deal with datasets that are, at the same time, high-dimensional and class-imbalanced. To make a contribution in this direction, our work presents a comparative study among different learning strategies that leverage both feature selection, to cope with high dimensionality, as well as cost-sensitive learning methods, to cope with class imbalance. Specifically, different ways of incorporating misclassification costs into the learning process have been explored. Also different feature selection heuristics have been considered, both univariate and multivariate, to comparatively evaluate their effectiveness on imbalanced data. The experiments have been conducted on three challenging benchmarks from the genomic domain, gaining interesting insight into the beneficial impact of combining feature selection and cost-sensitive learning, especially in the presence of highly skewed data distributions.

Cost-sensitive matrixized classification learning with information entropy

Classifier design is one of the most significant fields in pattern recognition. Most classifiers are measured by classification accuracy, which assumes that all the misclassification cost are the same. In the real world, different misclassifications usually bring different losses. Based on this fact, cost-sensitive learning is becoming a hot research area in pattern recognition. However, in cost-sensitive learning, examples costs are often difficult to achieve and usually decided by the authors experience. Hence, combining the cost-sensitive learning and matrixized learning thoughts, we propose a two-class cost-sensitive matrixized classification model based on information entropy called CsMatMHKS in this paper. The proposed CsMatMHKS introduces information entropy which can reveal the uncertainty of one sample into matrixized learning framework to decrease the total misclassification cost. The experimental results on the UCI datasets and image datasets indicate that the CsMatMHKS not only reduces the sum of classification costs but also has comparable classification accuracy.

Semi-supervised attribute reduction for interval data based on misclassification cost

Semi-supervised attribute reduction for interval data based on misclassification cost

COSLE: Cost sensitive loan evaluation for P2P lending

The loan evaluation is a fundamental task in peer-to-peer (P2P) lending. Effective loan evaluation can help lenders make informed investment decisions. Existing methods do not consider the return of loans in the core learning stage and thus fail to explore the relationship between the return of loans and their final loan payoff outcomes. In this study, we propose a systematic loan evaluation framework called COst Sensitive Loan Evaluation (COSLE). Specifically, we first develop an instance-aware misclassification cost (IMCO) matrix, which specifies personalized cost for each loan. Then, we present a differential labelling algorithm called DILA cost for assigning node labels and assessing the corresponding cost. By integrating these enhancements into the tree-induction process, we construct a node splitting measurement called COG index. It exploits the relationship between the return information and the final payoff outcome. Additionally, we design the LER evaluation metric to measure the ability of a loan evaluation model to increase the lender’s return. Finally, the COSLE is used to improve popular tree models. Extensive experiments based on the Lending Club dataset show that our COSLE can effectively increase the lender’s return.

Deep Learning Software Defect Prediction Methods for Cloud Environments Research

This paper provides an in-depth study and analysis of software defect prediction methods in a cloud environment and uses a deep learning approach to justify software prediction. A cost penalty term is added to the supervised part of the deep ladder network; that is, the misclassification cost of different classes is added to the model. A cost-sensitive deep ladder network-based software defect prediction model is proposed, which effectively mitigates the negative impact of the class imbalance problem on defect prediction. To address the problem of lack or insufficiency of historical data from the same project, a flow learning-based geodesic cross-project software defect prediction method is proposed. Drawing on data information from other projects, a migration learning approach was used to embed the source and target datasets into a Gaussian manifold. The kernel encapsulates the incremental changes between the differences and commonalities between the two domains. To this point, the subspace is the space of two distributional approximations formed by the source and target data transformations, with traditional in-project software defect classifiers used to predict labels. It is found that real-time defect prediction is more practical because it has a smaller amount of code to review; only individual changes need to be reviewed rather than entire files or packages while making it easier for developers to assign fixes to defects. More importantly, this paper combines deep belief network techniques with real-time defect prediction at a fine-grained level and TCA techniques to deal with data imbalance and proposes an improved deep belief network approach for real-time defect prediction, while trying to change the machine learning classifier underlying DBN for different experimental studies, and the results not only validate the effectiveness of using TCA techniques to solve the data imbalance problem but also show that the defect prediction model learned by the improved method in this paper has better prediction performance.

Applying machine learning algorithms to predict default probability in the online credit market: Evidence from China

Using data from Renrendai and three machine learning algorithms, namely, k-nearest neighbor, support vector machine, and random forest, we predicted the default probability of online loan borrowers and compared their prediction performance with that of a logistic model. The results show that, first, based on the AUC (area under the ROC curve) value, accuracy rate and Brier score, the machine learning models can accurately predict the default risk of online borrowers. Second, the integrated discrimination improvement (IDI) test results show that the prediction performance of the machine learning algorithms is significantly better than that of the logistic model. Third, after constructing the investor profit function with misclassification cost, we find that the machine learning algorithms can provide more benefits to investors.

Informative Gene Selection Based on Cost-Sensitive Fast Correlation- Based Filter Feature Selection

Background: Informative gene selection is an essential step in performing tumor classification. However, it is difficult to select informative genes related to tumors from large-scale gene expression profiles because of their characteristics, such as high dimensionality, relatively small samples, and class imbalance, and some genes are superfluous and irrelevant. Objective: Many researchers analyze and process gene expression data to obtain classified gene subsets by using machine learning methods. However, the gene expression profiles of tumors often have massive computational challenges. In addition, when improving feature importance and classification accuracy, cost estimation is often ignored in traditional feature selection algorithms, which makes tumor classification more difficult. Methods: In this study, a novel informative gene selection method based on cost-sensitive fast correlation- based filter (CS-FCBF) feature selection is proposed. Results: First, the symmetric uncertainty index is used to evaluate the correlation between informativeFirst, the symmetric uncertainty index is used to evaluate the correlation between informative genes and class labels, then a large number of irrelevant and redundant genes are quickly filtered according to importance. Thereby, a candidate gene subset is generated. Second, costsensitive learning, which introduces the misclassification cost matrix and support vector machine attribute evaluation, is used to obtain the top-ranked gene subset with minimum misclassification loss. Finally, the candidate gene subset is optimized. genes and class labels, then a large number of irrelevant and redundant genes are quickly filtered according to importance. Thereby, a candidate gene subset is generated. Second, costsensitive learning, which introduces the misclassification cost matrix and support vector machine attribute evaluation, is used to obtain the top-ranked gene subset with minimum misclassification loss. Finally, the candidate gene subset is optimized. Conclusion: This experiment was verified in eight independent tumor datasets. By comparing and analyzing CS-FCBF with another three hybrids of typical gene selection algorithms combined with cost-sensitive learning, we found that the method proposed in this study has a better classification performance with fewer selected genes, which might provide guidance in tumor diagnosis and research.

Bias correction for linear discriminant analysis

• We present an asymptotically exact estimator of the optimal intercept for the linear discriminant analysis (LDA). • The proposed accurate estimator is developed based on the theory of random matrices of increasing dimension. • The simple form of the estimator provides analytical insights into the mechanism of the proposed bias correction in LDA. • Analytical insights are complemented by empirical observations. Linear discriminant analysis (LDA) is perhaps one of the most fundamental statistical pattern recognition techniques. In this work, we explicitly present, for the first time, an asymptotically exact estimator of the LDA optimal intercept in terms of achieving the lowest overall risk in the classification of two multivariate Gaussian distributions with a common covariance matrix and arbitrary misclassification costs. The proposed estimator of the optimal bias term is developed based on the theory of random matrices of increasing dimension in which the observation dimension and the sample size tend to infinity while keeping their magnitudes comparable. The simple form of this estimator provides us with some analytical insights into the working mechanism of the bias correction in LDA. We then complement these analytical insights with numerical experiments. In particular, empirical results using real data show that insofar as the overall risk is concerned, the proposed bias-corrected form of LDA can outperform the conventional LDA classifier in a wide range of misclassification costs. At the same time, the superiority of the proposed form over LDA tends to be more evident as dimensionality or the ratio between class-specific costs increase.

Constrained Naïve Bayes with application to unbalanced data classification

The Naïve Bayes is a tractable and efficient approach for statistical classification. In general classification problems, the consequences of misclassifications may be rather different in different classes, making it crucial to control misclassification rates in the most critical and, in many realworld problems, minority cases, possibly at the expense of higher misclassification rates in less problematic classes. One traditional approach to address this problem consists of assigning misclassification costs to the different classes and applying the Bayes rule, by optimizing a loss function. However, fixing precise values for such misclassification costs may be problematic in realworld applications. In this paper we address the issue of misclassification for the Naïve Bayes classifier. Instead of requesting precise values of misclassification costs, threshold values are used for different performance measures. This is done by adding constraints to the optimization problem underlying the estimation process. Our findings show that, under a reasonable computational cost, indeed, the performance measures under consideration achieve the desired levels yielding a user-friendly constrained classification procedure.

RB-CCR: Radial-Based Combined Cleaning and Resampling algorithm for imbalanced data classification

Real-world classification domains, such as medicine, health and safety, and finance, often exhibit imbalanced class priors and have asynchronous misclassification costs. In such cases, the classification model must achieve a high recall without significantly impacting precision. Resampling the training data is the standard approach to improving classification performance on imbalanced binary data. However, the state-of-the-art methods ignore the local joint distribution of the data or correct it as a post-processing step. This can causes sub-optimal shifts in the training distribution, particularly when the target data distribution is complex. In this paper, we propose Radial-Based Combined Cleaning and Resampling (RB-CCR). RB-CCR utilizes the concept of class potential to refine the energy-based resampling approach of CCR. In particular, RB-CCR exploits the class potential to accurately locate sub-regions of the data-space for synthetic oversampling. The category sub-region for oversampling can be specified as an input parameter to meet domain-specific needs or be automatically selected via cross-validation. Our 5times 2 cross-validated results on 57 benchmark binary datasets with 9 classifiers show that RB-CCR achieves a better precision-recall trade-off than CCR and generally out-performs the state-of-the-art resampling methods in terms of AUC and G-mean.

An optimal weighted machine learning model for detecting financial fraud

ABSTRACT The purpose of this study is to predict enterprises’ financial fraud. After collecting financial data and employing feature selection methods, totally eight features are selected. We select two best-performance machine learning models according to five indicators including accuracy, recall, specificity, AUC and the misclassification cost. Besides, an optimal weighted machine learning model, based on the two best-performance models, is proposed and the results confirm its good performance in forecasting enterprises’ financial fraud.

Cross-Modal Cortical Activity in the Brain Can Predict Cochlear Implantation Outcome in Adults: A Machine Learning Study.

Objectives: Prediction of cochlear implantation (CI) outcome is often difficult because outcomes vary among patients. Though the brain plasticity across modalities during deafness is associated with individual CI outcomes, longitudinal observations in multiple patients are scarce. Therefore, we sought a prediction system based on cross-modal plasticity in a longitudinal study with multiple patients.METHODS: Classification of CI outcomes between excellent or poor was tested based on the features of brain cross-modal plasticity, measured using event-related responses and their corresponding electromagnetic sources. A machine learning estimation model was applied to 13 datasets from 3 patients based on linear supervised training. Classification efficiency was evaluated comparing prediction accuracy, sensitivity/specificity, total mis-classification cost, and training time among feature set conditions.Results:Combined feature sets with the sensor and source levels dramatically improved classification accuracy between excellent and poor outcomes. Specifically, the tactile feature set best explained CI outcome (accuracy, 98.83 ± 2.57%; sensitivity, 98.00 ± 0.01%; specificity, 98.15 ± 4.26%; total misclassification cost, 0.17 ± 0.38; training time, 0.51 ± 0.09 sec), followed by the visual feature (accuracy, 93.50 ± 4.89%; sensitivity, 89.17 ± 8.16%; specificity, 98.00 ± 0.01%; total misclassification cost, 0.65 ± 0.49; training time, 0.38 ± 0.50 sec).Conclusion:Individual tactile and visual processing in the brain best classified the current status when classified by combined sensor–source level features. Our results suggest that cross-modal brain plasticity due to deafness may provide a basis for classifying the status. We expect this novel method to contribute to the evaluation and prediction of CI outcomes.

Cost-sensitive meta-learning framework

PurposeThis paper aims to describe the use of a meta-learning framework for recommending cost-sensitive classification methods with the aim of answering an important question that arises in machine learning, namely, “Among all the available classification algorithms, and in considering a specific type of data and cost, which is the best algorithm for my problem?”Design/methodology/approachThis paper describes the use of a meta-learning framework for recommending cost-sensitive classification methods for the aim of answering an important question that arises in machine learning, namely, “Among all the available classification algorithms, and in considering a specific type of data and cost, which is the best algorithm for my problem?” The framework is based on the idea of applying machine learning techniques to discover knowledge about the performance of different machine learning algorithms. It includes components that repeatedly apply different classification methods on data sets and measures their performance. The characteristics of the data sets, combined with the algorithms and the performance provide the training examples. A decision tree algorithm is applied to the training examples to induce the knowledge, which can then be used to recommend algorithms for new data sets. The paper makes a contribution to both meta-learning and cost-sensitive machine learning approaches. Those both fields are not new, however, building a recommender that recommends the optimal case-sensitive approach for a given data problem is the contribution. The proposed solution is implemented in WEKA and evaluated by applying it on different data sets and comparing the results with existing studies available in the literature. The results show that a developed meta-learning solution produces better results than METAL, a well-known meta-learning system. The developed solution takes the misclassification cost into consideration during the learning process, which is not available in the compared project.FindingsThe proposed solution is implemented in WEKA and evaluated by applying it to different data sets and comparing the results with existing studies available in the literature. The results show that a developed meta-learning solution produces better results than METAL, a well-known meta-learning system.Originality/valueThe paper presents a major piece of new information in writing for the first time. Meta-learning work has been done before but this paper presents a new meta-learning framework that is costs sensitive.

Learning to Learn: Algorithmic Inspirations from Human Problem Solving

We harness the ability of people to perceive and interact with visual patterns in order to enhance the performance of a machine learning method. We show how we can collect evidence about how people optimize the parameters of an ensemble classification system using a tool that provides a visualization of misclassification costs. Then, we use these observations about human attempts to minimize cost in order to extend the performance of a state-of-the-art ensemble classification system. The study highlights opportunities for learning from evidence collected about human problem solving to refine and extend automated learning and inference.

A random forest classifier with cost-sensitive learning to extract urban landmarks from an imbalanced dataset

ABSTRACT Urban landmarks play an important role as spatial references in spatial cognition, navigation, map design and urban planning. However, the current landmark extraction methods do not consider the imbalance between the landmark and non-landmarknon-landmark samples in a dataset, so the extraction results are biased toward the class with the majority of sample data, resulting in poor classification performance for the class with the fewest sample data. This study introduces a random forest (RF) classifier combined with cost-sensitive learning to extract urban landmarks automatically from a basic spatial database. First, the optimal feature set is determined according to the importance of features. Next, a cost-sensitive RF algorithm is applied to extract landmarks, which determines the misclassification cost according to the class distribution, and each decision tree is weighted by the classification results. The method has good performance, with a recall and area under the ROC curve (AUC) greater than 90%, and the model is also applicable to small sample sets, which can reduce the cost of manual labor.

Adaptive cost-sensitive assignment method for power system transient stability assessment

In power system transient stability assessment (TSA), the misclassification cost of unstable samples is much more serious than that of stable ones. Besides, for both unstable samples and stable samples, the impacts of critical samples on the assessment rules are more important than those of non-critical ones. Thus, an improved cost-sensitive assignment (ICSA) method based on the fault severity is proposed. Larger cost coefficients are assigned to critical samples compared with non-critical samples. Furthermore, sufficiently learning the information of misclassified samples is helpful to modify the assessment rules quickly and accurately. Meanwhile, the assessment rules may be greatly disturbed due to the existence of abnormal samples among training samples. Therefore, on basis of the ICSA method, an adaptive cost-sensitive assignment (ACSA) method is presented in this paper. In the process of model training, the cost coefficients of misclassified training samples are adaptively adjusted to improve the assessment rules. Then, the impacts of the abnormal samples on the assessment rules are eliminated by assigning their cost coefficients to 0. Finally, simulation experiments are carried out in the IEEE 39-bus system and a realistic system. It can be shown that the proposed ACSA method has high classification accuracy and excellent generalization ability.

Key Technology Considerations in Developing and Deploying Machine Learning Models in Clinical Radiology Practice.

The use of machine learning to develop intelligent software tools for the interpretation of radiology images has gained widespread attention in recent years. The development, deployment, and eventual adoption of these models in clinical practice, however, remains fraught with challenges. In this paper, we propose a list of key considerations that machine learning researchers must recognize and address to make their models accurate, robust, and usable in practice. We discuss insufficient training data, decentralized data sets, high cost of annotations, ambiguous ground truth, imbalance in class representation, asymmetric misclassification costs, relevant performance metrics, generalization of models to unseen data sets, model decay, adversarial attacks, explainability, fairness and bias, and clinical validation. We describe each consideration and identify the techniques used to address it. Although these techniques have been discussed in prior research, by freshly examining them in the context of medical imaging and compiling them in the form of a laundry list, we hope to make them more accessible to researchers, software developers, radiologists, and other stakeholders.

Mgini - Improved Decision Tree using Minority Class Sensitive Splitting Criterion for Imbalanced Data of Covid-19

In the time of COVID-19, medical facilities struggling to fight against the pandemic. Most of the countries face a tough time fighting against this virus outbreak. Even developed countries are struggling to deal with this virus outbreak. Common problem countries face is a lack of medical staff and medical equipment. Machine learning has the potential to play an important role in a different area of medical facilities. With the help of the machine learning model, an effective diagnostic tool can be built which helps in the time of scarcity of medical staff. However medical data is imbalanced and this skew nature of data prevent machine learning algorithm from achieving high accuracy. To deal with this problem of imbalanced data, we proposed a modified decision tree algorithm that uses a minority sensitive Gini index called Mgini. In an imbalanced dataset of COVID-19, it is important to focus on the reduction of overall misclassification cost instead of trying improvement in accuracy value. Mgini is useful splitting criteria when the misclassification cost of the minority sample is huge as compared to the majority class. The use of this proposed new Gini index as a splitting criterion in the decision tree reduces the misclassification cost. Mgini based decision tree has higher accuracy and low misclassification cost as compare to the traditional Gini index based CART algorithm. Our proposed cost-sensitive approach improves imbalanced data classification without the use of data level sampling techniques.