Unequal Misclassification Costs Research Articles

Detection of paralytic shellfish toxins by near-infrared spectroscopy based on a near-Bayesian SVM classifier with unequal misclassification costs.

Paralytic shellfish poisoning caused by human consumption of shellfish fed on toxic algae is a public health hazard. It is essential to implement shellfish monitoring programs to minimize the possibility of shellfish contaminated by paralytic shellfish toxins (PST) reaching the marketplace. This paper proposes a rapid detection method for PST in mussels using near-infrared spectroscopy (NIRS) technology. Spectral data in the wavelength range of 950-1700 nm for PST-contaminated and non-contaminated mussel samples were used to build the detection model. Near-Bayesian support vector machines (NBSVM) with unequal misclassification costs (u-NBSVM) were applied to solve a classification problem arising from the fact that the quantity of non-contaminated mussels was far less than that of PST-contaminated mussels in practice. The u-NBSVM model performed adequately on imbalanced datasets by combining unequal misclassification costs and decision boundary shifts. The detection performance of the u-NBSVM did not decline as the number of PST samples decreased due to adjustments to the misclassification costs. When the number of PST samples was 20, the G-mean and accuracy reached 0.9898 and 0.9944, respectively. Compared with the traditional support vector machines (SVMs) and the NBSVM, the u-NBSVM model achieved better detection performance. The results of this study indicate that NIRS technology combined with the u-NBSVM model can be used for rapid and non-destructive PST detection in mussels. © 2023 Society of Chemical Industry.

Tailored Bayes: a risk modeling framework under unequal misclassification costs.

Risk prediction models are a crucial tool in healthcare. Risk prediction models with a binary outcome (i.e., binary classification models) are often constructed using methodology which assumes the costs of different classification errors are equal. In many healthcare applications, this assumption is not valid, and the differences between misclassification costs can be quite large. For instance, in a diagnostic setting, the cost of misdiagnosing a person with a life-threatening disease as healthy may be larger than the cost of misdiagnosing a healthy person as a patient. In this article, we present Tailored Bayes (TB), a novel Bayesian inference framework which "tailors" model fitting to optimize predictive performance with respect to unbalanced misclassification costs. We use simulation studies to showcase when TB is expected to outperform standard Bayesian methods in the context of logistic regression. We then apply TB to three real-world applications, a cardiac surgery, a breast cancer prognostication task, and a breast cancer tumor classification task and demonstrate the improvement in predictive performance over standard methods.

The OCS-SVM: An Objective-Cost-Sensitive SVM With Sample-Based Misclassification Cost Invariance

Studies on the traditional support vector machine (SVM) implicitly assume that the costs of different types of mistakes are the same and minimize the error rate. On the one hand, it is not enough for many practical applications to rely solely on the error rate, which reflects only the average classification ability of a classifier. It is also of great significance to consider the performance of classifiers from the perspective of each sample. On the other hand, many real-world problems, such as credit card fraud detection, intrusion detection, oil-spill detection and cancer diagnosis, usually involve substantially unequal misclassification costs. To solve this problem, many works on the cost-sensitive SVM (CS-SVM) have emerged. The misclassification costs for this model are generally provided by domain experts. Inspired by the concept of the CS-SVM, we propose a new SVM with sample-based misclassification cost invariance with the aim of constructing a relatively reliable classifier. The relatively reliable classifier is defined as the one with low probabilities of finding a classifier that correctly classifies each misclassified sample. Note that the cost is determined by the inherent characteristics of each sample rather than being subjectively assigned, so we denote the proposed classifier as the objective-cost-sensitive SVM (OCS-SVM). The experimental results demonstrate the superiority of the proposed method compared with nine other commonly used classifiers.

Improved Cost-Sensitive Support Vector Machine Classifier for Breast Cancer Diagnosis

As one of the most prevalent cancers among women worldwide, breast cancer has attracted the most attention by researchers. It has been verified that an accurate and early detection of breast cancer can increase the chances for the patients to take the right treatment plan and survive for a long time. Nowadays, numerous classification methods have been utilized for breast cancer diagnosis. However, most of these classification models have concentrated on maximum the classification accuracy, failed to take into account the unequal misclassification costs for the breast cancer diagnosis. To the best of our knowledge, misclassifying the cancerous patient as non-cancerous has much higher cost compared to misclassifying the non-cancerous as cancerous. Consequently, in order to tackle this deficiency and further improve the classification accuracy of the breast cancer diagnosis, we propose an improved cost-sensitive support vector machine classifier (ICS-SVM) for the diagnosis of breast cancer. In the proposed approach, we take full account of unequal misclassification costs of breast cancer intelligent diagnosis and provide more reasonable results over previous works and conventional classification models. To evaluate the performance of the proposed approach, Wisconsin Breast Cancer (WBC) and Wisconsin Diagnostic Breast Cancer (WDBC) breast cancer datasets obtained from the University of California at Irvine (UCI) machine learning repository have been studied. The experimental results demonstrate that the proposed hybrid algorithm outperforms all the existing methods. Promisingly, the proposed method can be regarded as a useful clinical tool for breast cancer diagnosis and could also be applied to other illness diagnosis.

The Reliability of Classification of Terminal Nodes in GUIDE Decision Tree to Predict the Nonalcoholic Fatty Liver Disease.

Tree structured modeling is a data mining technique used to recursively partition a dataset into relatively homogeneous subgroups in order to make more accurate predictions on generated classes. One of the classification tree induction algorithms, GUIDE, is a nonparametric method with suitable accuracy and low bias selection, which is used for predicting binary classes based on many predictors. In this tree, evaluating the accuracy of predicted classes (terminal nodes) is clinically of special importance. For this purpose, we used GUIDE classification tree in two statuses of equal and unequal misclassification cost in order to predict nonalcoholic fatty liver disease (NAFLD), considering 30 predictors. Then, to evaluate the accuracy of predicted classes by using bootstrap method, first the classification reliability in which individuals are assigned to a unique class and next the prediction probability reliability as support for that are considered.

Near-Bayesian Support Vector Machines for imbalanced data classification with equal or unequal misclassification costs

Support Vector Machines (SVMs) form a family of popular classifier algorithms originally developed to solve two-class classification problems. However, SVMs are likely to perform poorly in situations with data imbalance between the classes, particularly when the target class is under-represented. This paper proposes a Near-Bayesian Support Vector Machine (NBSVM) for such imbalanced classification problems, by combining the philosophies of decision boundary shift and unequal regularization costs. Based on certain assumptions which hold true for most real-world datasets, we use the fractions of representation from each of the classes, to achieve the boundary shift as well as the asymmetric regularization costs. The proposed approach is extended to the multi-class scenario and also adapted for cases with unequal misclassification costs for the different classes. Extensive comparison with standard SVM and some state-of-the-art methods is furnished as a proof of the ability of the proposed approach to perform competitively on imbalanced datasets. A modified Sequential Minimal Optimization (SMO) algorithm is also presented to solve the NBSVM optimization problem in a computationally efficient manner.

Relative cost curves: An alternative to AUC and an extension to 3-class problems

to specify exactly the consequences or costs of misclassifications. This paper is devoted to Relative Cost Curves (RCC) ‐ a graphical technique for visualising the performance of binary classifiers over the full range of possible relative misclassification costs. This curve provides helpful information to choose the best set of classifiers or to estimate misclassification costs if those are not known precisely. In this paper, the concept of Area Above the RCC (AAC) is introduced, a scalar measure of classifier performance under unequal misclassification costs problem. We also extend RCC to multicategory problems when misclassification costs depend only on the true class.

Comparing the Performance of Zero Mean Classification Functions under Unequal Misclassification Cost

In this study the performance of Minimum Expected Cost of Misclassification method (MECM) and Quadratic Discriminant Function approach (QDF) were compared and evaluated for the case of equal mean discrimination under unequal misclassification cost. 30 pairs of Female liked sex twins extracted from Stocks (1933) ten (10) variate data on 832 twin children was used for evaluation. Discriminant functions were derived under each of the following misclassification cost ratios; 1: 1, 1: 2, 1: 3 and 1: 4 and their error rate estimates determined using the Cross Validation (CV) and Balanced Error Rate (BER) methods. Least Mean error rates were recorded under QDF method as compared to that of MECM. The error rate estimates showed the QDF outperforming the MECM in the provision of maximum separation between the two groups. Also the two classification rules were found to be sensitive to misclassification cost ratios exceeding 1:2.

Minimum-risk training for semi-Markov conditional random fields with application to handwritten Chinese/Japanese text recognition

Semi-Markov conditional random fields (semi-CRFs) are usually trained with maximum a posteriori (MAP) criterion which adopts the 0/1 cost for measuring the loss of misclassification. In this paper, based on our previous work on handwritten Chinese/Japanese text recognition (HCTR) using semi-CRFs, we propose an alternative parameter learning method by minimizing the risk on the training set, which has unequal misclassification costs depending on the hypothesis and the ground-truth. Based on this framework, three non-uniform cost functions are compared with the conventional 0/1 cost, and training data selection is incorporated to reduce the computational complexity. In experiments of online handwriting recognition on databases CASIA-OLHWDB and TUAT Kondate, we compared the performances of the proposed method with several widely used learning criteria, including conditional log-likelihood (CLL), softmax-margin (SMM), minimum classification error (MCE), large-margin MCE (LM-MCE) and max-margin (MM). On the test set (online handwritten texts) of ICDAR 2011 Chinese handwriting recognition competition, the proposed method outperforms the best system in competition.

Mining Life Insurance Data for Customer Attrition Analysis

Customer attrition is an increasingly pressing issue faced by many insurance providers today. Retaining customers who purchase life insurance policies is an even bigger challenge since the policy duration spans for more than twenty years. Companies are eager to reduce these attrition rates in the customer-base by analyzing operational data. Data mining techniques play an important role in facilitating these retention efforts. The objective of this study is to analyse customer attrition by classifying all policy holders who are likely to terminate their policies. These customers who are at high risk of attrition can then be targeted for promotions to reduce the rate of attrition. Data mining techniques such as Decision trees and Neural Networks are employed in this study. Models generated are evaluated using ROC curves and AUC values. Our research also adopts cost sensitive learning strategies to address issues such as imbalanced class labels and unequal misclassification costs. 

Calibrated asymmetric surrogate losses

Surrogate losses underlie numerous state-of-the-art binary classification algorithms, such as support vector machines and boosting. The impact of a surrogate loss on the statistical performance of an algorithm is well-understood in symmetric classification settings, where the misclassification costs are equal and the loss is a margin loss. In particular, classification-calibrated losses are known to imply desirable properties such as consistency. While numerous efforts have been made to extend surrogate loss-based algorithms to asymmetric settings, to deal with unequal misclassification costs or training data imbalance, considerably less attention has been paid to whether the modified loss is still calibrated in some sense. This article extends the theory of classification-calibrated losses to asymmetric problems. As in the symmetric case, it is shown that calibrated asymmetric surrogate losses give rise to excess risk bounds, which control the expected misclassification cost in terms of the excess surrogate risk. This theory is illustrated on the class of uneven margin losses, and the uneven hinge, squared error, exponential, and sigmoid losses are treated in detail.

Multi-class HingeBoost

Multi-class molecular cancer classification has great potential clinical implications. Such applications require statistical methods to accurately classify cancer types with a small subset of genes from thousands of genes in the data. This paper presents a new functional gradient descent boosting algorithm that directly extends the HingeBoost algorithm from the binary case to the multi-class case without reducing the original problem to multiple binary problems. Minimizing a multi-class hinge loss with boosting technique, the proposed HingeBoost has good theoretical properties by implementing the Bayes decision rule and providing a unifying framework with either equal or unequal misclassification costs. Furthermore, we propose Twin HingeBoost which has better feature selection behavior than HingeBoost by reducing the number of ineffective covariates. Simulated data, benchmark data and two cancer gene expression data sets are utilized to evaluate the performance of the proposed approach. Simulations and the benchmark data showed that the multi-class HingeBoost generated accurate predictions when compared with the alternative methods, especially with high-dimensional covariates. The multi-class HingeBoost also produced more accurate prediction or comparable prediction in two cancer classification problems using gene expression data. This work has shown that the HingeBoost provides a powerful tool for multi-classification problems. In many applications, the classification accuracy and feature selection behavior can be further improved when using Twin HingeBoost.

Boosting the Generalized Margin in Cost-Sensitive Multiclass Classification

The boosting algorithm is one of the most successful binary classification techniques due to its relative immunity to overfitting and flexible implementation. Several attempts have been made to extend the binary boosting algorithm to multiclass classification. In this article, a novel cost-sensitive multiclass boosting algorithm is proposed that naturally extends the popular binary AdaBoost algorithm and admits unequal misclassification costs. The proposed multiclass boosting algorithm achieves superior classification performance by combining weak candidate models that only need to be better than random guessing. More importantly, the proposed algorithm achieves a large margin separation of the training sample while attaining an L1-norm constraint on the model complexity. Finally, the effectiveness of the proposed algorithm is demonstrated in a number of simulated and real experiments. The supplementary files are available online, including the technical proofs, the implemented R code, and the real datasets.

Cost-sensitive case-based reasoning using a genetic algorithm: Application to medical diagnosis

The paper studies the new learning technique called cost-sensitive case-based reasoning (CSCBR) incorporating unequal misclassification cost into CBR model. Conventional CBR is now considered as a suitable technique for diagnosis, prognosis and prescription in medicine. However it lacks the ability to reflect asymmetric misclassification and often assumes that the cost of a positive diagnosis (an illness) as a negative one (no illness) is the same with that of the opposite situation. Thus, the objective of this research is to overcome the limitation of conventional CBR and encourage applying CBR to many real world medical cases associated with costs of asymmetric misclassification errors. The main idea involves adjusting the optimal cut-off classification point for classifying the absence or presence of diseases and the cut-off distance point for selecting optimal neighbors within search spaces based on similarity distribution. These steps are dynamically adapted to new target cases using a genetic algorithm. We apply this proposed method to five real medical datasets and compare the results with two other cost-sensitive learning methods-C5.0 and CART. Our finding shows that the total misclassification cost of CSCBR is lower than other cost-sensitive methods in many cases. Even though the genetic algorithm has limitations in terms of unstable results and over-fitting training data, CSCBR results with GA are better overall than those of other methods. Also the paired t-test results indicate that the total misclassification cost of CSCBR is significantly less than C5.0 and CART for several datasets. We have proposed a new CBR method called cost-sensitive case-based reasoning (CSCBR) that can incorporate unequal misclassification costs into CBR and optimize the number of neighbors dynamically using a genetic algorithm. It is meaningful not only for introducing the concept of cost-sensitive learning to CBR, but also for encouraging the use of CBR in the medical area. The result shows that the total misclassification costs of CSCBR do not increase in arithmetic progression as the cost of false absence increases arithmetically, thus it is cost-sensitive. We also show that total misclassification costs of CSCBR are the lowest among all methods in four datasets out of five and the result is statistically significant in many cases. The limitation of our proposed CSCBR is confined to classify binary cases for minimizing misclassification cost because our proposed CSCBR is originally designed to classify binary case. Our future work extends this method for multi-classification which can classify more than two groups.

HingeBoost: ROC-Based Boost for Classification and Variable Selection

In disease classification, a traditional technique is the receiver operative characteristic (ROC) curve and the area under the curve (AUC). With high-dimensional data, the ROC techniques are needed to conduct classification and variable selection. The current ROC methods do not explicitly incorporate unequal misclassification costs or do not have a theoretical grounding for optimizing the AUC. Empirical studies in the literature have demonstrated that optimizing the hinge loss can maximize the AUC approximately. In theory, minimizing the hinge rank loss is equivalent to minimizing the AUC in the asymptotic limit. In this article, we propose a novel nonparametric method HingeBoost to optimize a weighted hinge loss incorporating misclassification costs. HingeBoost can be used to construct linear and nonlinear classifiers. The estimation and variable selection for the hinge loss are addressed by a new boosting algorithm. Furthermore, the proposed twin HingeBoost can select more sparse predictors. Some properties of HingeBoost are studied as well. To compare HingeBoost with existing classification methods, we present empirical study results using data from simulations and a prostate cancer study with mass spectrometry-based proteomics.

Incorporate Cost Matrix into Learning Vector Quantization Modeling: a Comparative Study of Genetic Algorithm, Simulated Annealing and Particle Swarm Optimization

Cost-sensitive learning is an important topic in bankruptcy prediction concerning the unequal misclassification cost of different classes. Learning vector quantization (LVQ) is a powerful tool to solve bankruptcy prediction problem as a classification task. The heuristic algorithms are applied widely in conjunction with artificial intelligent methods for solving optimization problems. The hybridization of heuristic techniques with existing classification algorithms is well illustrated in the field of bankruptcy prediction. In this paper, three hybrid heuristic-based LVQ approaches which combine LVQ with genetic algorithm, simulated annealing and particle swarm optimization respectively, are proposed to minimize the total misclassified cost under the asymmetric cost preference. The idea behind the hybrid classifier is the adoption of heuristic algorithms for the determination of the connection weights of the LVQ network. Experiments on French private company data show the proposed approaches offer interesting and viable alternatives for predictive reinforcement in cost-sensitive context.

An investigation of neural network classifiers with unequal misclassification costs and group sizes

Despite a larger number of successful applications of artificial neural networks for classification in business and other areas, published research has not considered the effects of misclassification costs and group sizes. Without the consideration of uneven misclassification costs, the classifier development will be compromised in minimizing the total misclassification errors. The use of this simplified model will not only result in poor decision capability when misclassification errors are significantly unequal, but also increase the model bias in favor of larger groups. This paper explores the issues of asymmetric misclassification costs and imbalanced group sizes through an application of neural networks to thyroid disease diagnosis. The results show that both asymmetric misclassification costs and imbalanced group sizes have significant effects on the neural network classification performance. In addition, we find that increasing the sample size and resampling are two effective approaches to counteract the problems.

Cost-Effective Hidden Markov Model-Based Image Segmentation

Image segmentation is an important preprocessing step in a sophisticated and complex image processing algorithm. In segmenting real-world images, the cost of misclassification could depend on the true class. For example, in a two-class (negative or positive class) problem, the cost of misclassifying positive to negative class could not be equal to that of misclassifying negative to positive class. However, existing algorithms do not take into account the unequal misclassification cost. In this letter, motivated by recent advances in machine learning theory, we introduce a procedure to minimize the misclassification cost with class-dependent cost. The procedure assumes the hidden Markov model (HMM) which has been popularly used for image segmentation in recent years. We represent all feasible HMM-based segmenters (or classifiers) as a set of points in the receiver operating characteristic (ROC) space. Then, the optimal segmenter (or classifier) is found by computing the tangential point between the iso-cost line with given slope and the convex hull of the feasible set in the ROC space. We illustrate the procedure by segmenting aerial images with different selection of misclassification costs.

A joint investigation of misclassification treatments and imbalanced datasets on neural network performance

Two important factors that impact a classification model’s performance are imbalanced data and unequal misclassification cost consequences. These are especially important considerations for neural network models developed to estimate the posterior probabilities of group membership used in classification decisions. This paper explores the issues of asymmetric misclassification costs and unbalanced group sizes on neural network classification performance using an artificial data approach that is capable of generating more complex datasets than used in prior studies and which adds new insights to the problem and the results. A different performance measure, that is capable of directly measuring classification performance consistency with Bayes decision rule, is used. The results show that both asymmetric misclassification costs and imbalanced group sizes have significant effects on neural network classification performance both independently and via interaction effects. These are not always intuitive; they supplement prior findings, and raise issues for the future.

Assessing the stability of classification trees using Florida birth data

Using 1998 and 1999 singleton birth data of the State of Florida, we study the stability of classification trees. Tree stability depends on both the learning algorithm and the specific data set. In this study, test samples are used in statistical learning to evaluate both stability and predictive performance. We also use the resampling technique bootstrap, which can be regarded as data self-perturbation, to evaluate the sensitivity of the modeling algorithm with respect to the specific data set. We demonstrate that the selection of the cost function plays an important role in stability. In particular, classifiers with equal misclassification costs and equal priors are less stable compared to those with unequal misclassification costs and equal priors.