Misclassification Costs Research Articles

Geometric relative margin machine for heterogeneous distribution and imbalanced classification

This paper introduces the geometric relative margin machine (GRMM) to address class imbalance and heterogeneous data distributions in classification tasks. GRMM innovatively incorporates boundary information as distribution information and implements differentiated misclassification cost strategies for each class, allowing adaptive adjustments to the decision hyperplane and enhancing robustness against outliers. The model employs a block iterative algorithm for optimization. Experimental results on artificial, benchmark, and credit risk assessment datasets demonstrate that GRMM significantly outperforms other benchmark methods in terms of classification accuracy and robustness. This research provides a comprehensive and effective solution to imbalanced classification problems, with potential applications in real-world scenarios, such as credit assessment.

Support Vector Machine-Based Fault Diagnosis under Data Imbalance with Application to High-Speed Train Electric Traction Systems

The safety and reliability of high-speed train electric traction systems are crucial. However, the operating environment for China Railway High-speed (CRH) trains is challenging, with severe working conditions. Dataset imbalance further complicates fault diagnosis. Therefore, conducting fault diagnosis for high-speed train electric traction systems under data imbalance is not only theoretically important but also crucial for ensuring vehicle safety. Firstly, when addressing the data imbalance issue, the fault diagnosis mechanism based on support vector machines tends to prioritize the majority class when constructing the classification hyperplane. This frequently leads to a reduction in the recognition rate of minority-class samples. To tackle this problem, a self-tuning support vector machine is proposed in this paper by setting distinct penalty factors for each class based on sample information. This approach aims to ensure equal misclassification costs for both classes and achieve the objective of suppressing the deviation of the classification hyperplane. Finally, simulation experiments are conducted on the Traction Drive Control System-Fault Injection Benchmark (TDCS-FIB) platform using three different imbalance ratios to address the data imbalance issue. The experimental results demonstrate consistent misclassification costs for both the minority- and majority-class samples. Additionally, the proposed self-tuning support vector machine effectively mitigates hyperplane deviation, further confirming the effectiveness of this fault diagnosis mechanism for high-speed train electric traction systems.

MULTICR: Predicting Merged and Abandoned Code Changes in Modern Code Review Using Multi-Objective Search

Modern Code Review (MCR) is an essential process in software development to ensure high-quality code. However, developers often spend considerable time reviewing code changes before being merged into the main code base. Previous studies attempted to predict whether a code change was going to be merged or abandoned soon after it was submitted to improve the code review process. However, these approaches require complex cost-sensitive learning, which makes their adoption challenging since it is difficult for developers to understand the main factors behind the models’ predictions. To address this issue, we introduce in this paper, MULTICR, a multi-objective search-based approach that uses Multi-Objective Genetic Programming (MOGP) to learn early code review prediction models as IF-THEN rules. MULTICR evolves predictive models while maximizing the accuracy of both merged and abandoned classes, eliminating the need for misclassification cost estimation. To evaluate MULTICR, we conducted an empirical study on 146,612 code reviews from Eclipse, LibreOffice, and Gerrithub. The obtained results show that MULTICR outperforms existing baselines in terms of Mathew correlation coefficient (MCC) and F1 scores while learning less complex models compared to decision trees. Our experiments showed also how MULTICR allows identifying the main factors related to abandoned code reviews as well as their associated thresholds, making it a promising approach for early code review prediction with notable performance and interoperability. Additionally, we qualitatively evaluate MULTICR by conducting a user study through semi-structured interviews involving 10 practitioners from different organizations. The obtained results indicate that 90% of the participants find that MULTICR is useful and can help them to improve the code review process. Additionally, the learned IF-THEN rules of MULTICR are transparent.

Splitting long‐term and short‐term financial ratios for improved financial distress prediction: Evidence from Taiwanese public companies

AbstractFinancial distress occurs when a company cannot meet its financial obligations within a specified timeframe, often owing to prolonged poor operational performance. While studies on financial distress prediction (FDP) use financial ratios (FRs) to forecast distress, they neglect to differentiate long‐term (LT) attributes from FRs. To address this gap, our study introduces a novel model that distinguishes between LT and short‐term (ST) accounting attributes in FRs. Using data from Taiwanese public companies (1991–2018), our proposed model employs a stacking ensemble classifier to split LT and ST Altman's ratios. This study addresses three key questions: (1) Do models involving split of LT and ST ratios outperform those that combine them? (2) How reliable and robust are these proposed models? (3) What is the proposed model's impact on distress prediction? The results show a significant outperformance of the existing solution, with higher accuracy, lower Type I and Type II errors, and reduced misclassification costs. These models are reliable in handling imbalanced data, proving suitable for real‐market investigations. Diverse FR contexts from previous Taiwanese studies validate the distinction between LT and ST features, representing robust performance. This model identifies characteristics of correctly and incorrectly predicted distress in companies, providing nuanced insights into complex distress attributes. This study introduces a pioneering model demonstrating superior predictive accuracy, reliability, and robustness by considering the split between LT and ST accounting attributes. It lays a foundation for future studies to extend and refine the proposed model, offering valuable insights into the complex dynamics of FDP.

Validity of luminometry and bacteriological tests for diagnosing intramammary infection at dry-off in dairy cows

The objective of this cross-sectional study was to estimate the validity of laboratory culture, Petrifilm and Tri-Plate on-farm culture systems, and luminometry to correctly identify IMI at dry-off in dairy cows, considering all tests as imperfect. From September 2020 until December 2021, we collected composite milk samples from cows before dry-off and divided them into 4 aliquots for the luminometry test, the Petrifilm (aerobic count), the Tri-Plate, and the laboratory culture. While we assessed multiple thresholds of relative light units (RLU) for the luminometry, we used thresholds of ≥100 cfu/mL for the laboratory culture, ≥ 50 cfu/mL for the Petrifilm, and ≥1 cfu for the Tri-Plate. We fitted Bayesian latent class analysis (LCA) models to estimate the sensitivity (Se) and specificity (Sp) for each test to identify IMI, with 95% credibility interval (BCI). Using different prevalence measures (0.30, 0.50, and 0.70), we calculated the predictive values (PV) and misclassification cost terms (MCT) at different false-negative to false-positive ratios (FN:FP). A total of 333 cows were enrolled in the study from one commercial Holstein herd. The validity of the luminometry was poor for all thresholds, with Se of 0.51 (95% BCI = 0.43-0.59) and Sp of 0.38 (95% BCI = 0.26-0.50) when using a threshold of ≥150 RLU. The laboratory culture had Se of 0.93 (95% BCI = 0.85-0.98) and Sp of 0.69 (95% BCI = 0.49-0.89), the Petrifilm had Se of 0.91 (95% BCI = 0.80-0.98) and Sp of 0.71 (95% BCI = 0.51-0.90), and the Tri-Plate had Se of 0.65 (95% BCI = 0.53-0.82) and Sp of 0.85 (95% BCI = 0.66-0.97). Bacteriological tests had good PVs, with comparable positive PV for all 3 tests, but lower negative PV for the Tri-Plate compared with the laboratory culture and the Petrifilm. For a prevalence of IMI of 0.30, all 3 tests had similar MCT, but for prevalence of 0.50 and 0.70, the Tri-Plate had higher MCT in scenarios where leaving a cow with IMI untreated is considered to have greater detrimental impacts than treating a healthy cow (i.e., FN:FP of 3:1). Our results showed that the bacteriological tests have adequate validity to diagnose IMI at dry-off, but the luminometry does not. We concluded that, while luminometry is not useful to identify IMI at dry-off, the Petrifilm and Tri-Plate tests performed similarly to the laboratory culture, depending on the prevalence and the importance of the FP and FN results.

A novel cost-sensitive three-way intuitionistic fuzzy large margin classifier

Three-way decision (3WD) has been widely applied in diverse fields in tackling uncertainty, particularly in classification domain. As a discriminative learning algorithm, Large Margin Distribution Machine (LDM) aims to maximize the inter-class margin by leveraging the marginal distribution of samples, which captures the underlying structure and achieves superior classification performance. However, existing LDMs still suffer from some inherent flaws, including: i) the neglect of fuzzy decision items, leading to the inadequate handling of uncertainty; ii) the utilization of a cost-insensitive mechanism, resulting in high misclassification costs; and iii) the disregard of sample credibility, contributing to the noise susceptibility. To address these limitations, we introduce the intuitionistic fuzzy (IF) and cost-sensitive 3WD (CS3WD), presenting an innovative model known as the CS3W-IFLMC model. The IF theory is employed to quantify sample confidence levels, augmenting noise suppression in our proposed model. Moreover, the integration of the CS3WD method effectively reduces the overall decision cost and further improves the handling of uncertainty in datasets. Consequently, the CS3W-IFLMC model demonstrates superior noise resilience and generalization performance. Our comparative experiments validate the efficacy of the proposed CS3W-IFLMC model in achieving robustness to noise while upholding a competitive classification performance.

Enhanced Cost-sensitive Ensemble Learning for Imbalanced Class in Medical Data

The challenge of imbalanced datasets in machine learning, particularly in the medical and public health sectors, necessitates innovative solutions that enhance predictive accuracy and reduce misclassification costs. This paper introduces the Enhanced Cost Sensitive Ensemble Learning (ECSEL), a novel approach that combines cost-sensitive learning with ensemble techniques to address the imbalance inherent in many critical datasets. We evaluate ECSEL on two specific datasets: the Framingham Heart Study dataset, which is pivotal in cardiovascular disease prediction, and a dataset focused on COVID-19 infection forecasting, relevant for public health responses. The results demonstrate that ECSEL significantly outperforms traditional methods like SMOTE, D-SMOTE, and BP-SMOTE by improving accuracy, precision, recall, and ROC-AUC values, particularly reducing Type II errors in scenarios where the cost of false negatives is exceptionally high. The method’s effectiveness is showcased through its superior performance in predicting cardiovascular risks and its robustness in forecasting the spread of COVID-19, reflecting its applicability and potential in real-world settings.

Reject inference in credit scoring based on cost-sensitive learning and joint distribution adaptation method

As traditional credit evaluation methods generally only use accepted sample modeling, the rejected data is omitted, which means the model's prediction of new customers is biased. However, reject inference can be used to solve this credit evaluation sample selection bias. This paper proposes a new reject inference method based on joint distribution adaptation (JDA) and cost-sensitive semi-supervised support vector machines (CS4VM). First, this method uses both accepted (labeled) samples and rejected (unlabeled) samples modeling, which overcomes the deviations in traditional credit evaluation methods. Second, as the accepted sample and the rejected sample distributions are different, this method reduces the distribution differences between the accepted and rejected sample sets, which ensures that the sample data conforms to the basic assumptions in the semi-supervised model, and improves the performance of the classification model. Third, this method reduces the overall cost in the actual credit business by considering both the traditional misclassification costs when mining the default samples and the different decision weights for the accepted and rejected samples. Finally, an empirical study verifies the excellent predictive performance of the proposed method and effectively reduces the total credit costs.

Learning Only When It Matters: Cost-Aware Long-Tailed Classification

Most current long-tailed classification approaches assume the cost-agnostic scenario, where the training distribution of classes is long-tailed while the testing distribution of classes is balanced. Meanwhile, the misclassification costs of all instances are the same. On the other hand, in many real-world applications, it is more proper to assume that the training and testing distributions of classes are the same, while the misclassification cost of tail-class instances is varied. In this work, we model such a scenario as cost-aware long-tailed classification, in which the identification of high-cost tail instances and focusing learning on them thereafter is essential. In consequence, we propose the learning strategy of augmenting new instances based on adaptive region partition in the feature space. We conduct theoretical analysis to show that under the assumption that the feature-space distance and the misclassification cost are correlated, the identification of high-cost tail instances can be realized by building region partitions with a low variance of risk within each region. The resulting AugARP approach could significantly outperform baseline approaches on both benchmark datasets and real-world product sales datasets.

MVQS: Robust multi-view instance-level cost-sensitive learning method for imbalanced data classification

Multi-view imbalanced learning is to handle the datasets with multi-view representations and imbalanced classes. Existing multi-view imbalanced learning methods can be divided into two main categories: multi-view ensemble learning and multi-view cost-sensitive learning. However, these methods suffer from the following problems: 1) neglecting either consensus or complementary information, 2) complex preprocessing and information fusion in multi-view ensemble learning and manual assignment of misclassification costs in multi-view cost-sensitive learning, and 3) limited ability to handle noisy samples. Therefore, we aim to design a concise and unified framework to grapple with the multi-view representations, imbalanced classes and noisy samples simultaneously. Inspired by the merits of support vector machine (SVM) and quadratic type squared error (QTSE) loss function, we propose a robust multi-view instance-level cost-sensitive SVM with QTSE loss (MVQS) for imbalanced data classification. The consensus regularization term and combination weight strategy are employed to fully exploit multi-view information. The QTSE loss can adaptively impose instance-level penalties to the misclassification of samples, and make MVQS be robust to noisy samples. We solve MVQS with the alternating direction method of multipliers (ADMM) and the gradient descent (GD) algorithm. Comprehensive experiments validate that MVQS is more competitive and robust than other benchmark approaches.

Optimizing Ensemble Learning to Reduce Misclassification Costs in Credit Risk Scorecards

Credit risk scorecard models are utilized by lending institutions to optimize decisions on credit approvals. In recent years, ensemble learning has often been deployed to reduce misclassification costs in credit risk scorecards. In this paper, we compared the risk estimation of 26 widely used machine learning algorithms based on commonly used statistical metrics. The best-performing algorithms were then used for model selection in ensemble learning. For the first time, we proposed financial criteria that assess the impact of losses associated with both false positive and false negative predictions to identify optimal ensemble learning. The German Credit Dataset (GCD) is augmented with simulated financial information according to a hypothetical mortgage portfolio observed in UK, European and Australian banks to enable the assessment of losses arising from misclassification costs. The experimental results using the simulated GCD show that the best predictive individual algorithm with the accuracy of 0.87, Gini of 0.88 and Area Under the Receiver Operating Curve of 0.94 was the Generalized Additive Model (GAM). The ensemble learning method with the lowest misclassification cost was the combination of Random Forest (RF) and K-Nearest Neighbors (KNN), totaling USD 417 million in costs (USD 230 for default costs and USD 187 for opportunity costs) compared to the costs of the GAM (USD 487, USD 287 and USD 200). Implementing the proposed financial criteria has led to a significant USD 70 million reduction in misclassification costs derived from a small sample. Thus, the lending institutions’ profit would considerably rise as the number of submitted credit applications for approval increases.

An Application of Multinomial Mis-Classification Cost Matrix For A P- To – P Lending Credit Score

An emerging new form of online credit for lending, different from traditional sources of finance, such as banks and building societies, where lenders provide loans to borrowers directly is termed as P-P. Many of these credits are unsecured personal loans, thus credit score of loans is vital to regulate the default risk and improve profit for lenders and platforms. Standard two-fold classifiers may not be appropriate in this lending since there are multiple credit classes and misclassification costs vary largely across classes in the lending platforms. Cost Sensitive Classifiers have been studied extensively in this set of lending, but none of them have analyzed this issue from the perspective of multinomial classifications and measured the misclassification costs of different credit grades using actual losses and opportunity costs. The research intends to model credit score in p-p lending as a cost-sensitive multinomial classification problem. A misclassification cost matrix is proposed for credit scoring with a set of equations and models to estimate the costs. A replication study using a publicly available data is conducted to evaluate the performance and validate the usefulness of the proposed misclassification cost matrix with the help of an R statistical package developed to aid the application of the model. The outcomes showed that the cost-sensitive multinomial classifiers can significantly decrease the total cost, which is vital for the p-p survival and profitability.

Examining different cost ratio frameworks for decision rule machine learning algorithms in diagnostic application.

Artificial Intelligence (AI) plays a pivotal role in the diagnosis of health conditions ranging from general well-being to critical health issues. In the realm of health diagnostics, an often overlooked but critical aspect is the consideration of cost-sensitive learning, a facet that this study prioritizes over the non-invasive nature of the diagnostic process whereas the other standard metrics such as accuracy and sensitivity reflect weakness in error profile. This research aims to investigate the total cost of misclassification (Total Cost) by decision rule Machine Learning (ML) algorithms implemented in Java platforms such as DecisionTable, JRip, OneR, and PART. An augmented dataset with conjunctiva images along candidates' demographic and anthropometric features under supervised learning is considered with a specific emphasis on cost-sensitive classification. The opted decision rule classifiers use the text features, additionally the image feature 'a* value of CIELAB color space' extracted from the conjunctiva digital images as input attributes. The pre-processing consists of amalgamating text and image features on a uniform scale, normalizing. Then the 10-fold cross-validation enables the classification of samples into two categories: the presence or absence of the anemia. This study utilizes the Cost Ratio (ρ) extracted from the cost matrix to meticulously monitor the Total Cost in four different cost ratio methodologies namely Uniform (U), Uniform Inverted (UI), Non-Uniform (NU), and Non-Uniform Inverted (NUI). It has been established that the PART classifier stands out as the top performer in this binary classification task, yielding the lowest mean total cost of 629.9 compared to other selected classifiers. Moreover, it demonstrates a comparatively lower standard deviation 335.9, and lower total cost across all four different cost ratio methodologies. The ranking of algorithm performance goes as follows: PART, JRIP, DecisionTable, and OneR. The significance of adopting a cost-sensitive learning approach is emphasized showing the PART classifier's consistent performance within the proposed framework for learning the anemia dataset. This emphasis on cost-sensitive learning not only enhances the recommendations in diagnosis but also holds the potential for substantial cost savings and makes it a noteworthy focal point in the advancement of AI-driven health care.

Decision trees applied to injury severity in road accidents with only one victim – focus on vulnerable users

The injury severity resulting from a road accident is usually defined based on the most serious victim, which can involve more than one driver, pedestrian, or passenger. Unlike other works that focus on driver characteristics, the present study developed for urban roads focuses on vulnerable users by limiting the database to accidents with a single victim with more serious injuries. Using the CART algorithm, the selected methodology strives to reduce the effects of data imbalance, incorporating various misclassification costs to improve the ‘accident with death’ prediction (minority class) and evaluating the quality results by association rules. After incorporating a 5:1 misclassification cost, recall increased from 0% to 43.3%. Two predictive ‘death’ decision rules were validated, and the identified factors were alcohol/drug consumption, type of accident, age group and annual average daily traffic for traffic above 70 km/h. Limiting the study to accidents with only one person suffering a more serious injury allowed for the identification of significant variables related to vulnerable users, and that can be considered in support of decision-making to reduce the severity of accidents.

A nested decision tree for event detection in smart grids

Digitalization process experienced by traditional power networks towards smart grids extend the challenges faced by power grid operators to the field of cybersecurity. False data injection attacks, one of the most common cyberattacks in smart grids, could lead the power grid to sabotage itself. In this paper, an event detection algorithm for cyberattack in smart grids is developed based on a decision tree. In order to find the most accurate algorithm, two different decision trees with two different goals have been trained: one classifies the status of the network, corresponding to an event, and the other will classify the location where the event is detected. To train the decision trees, a dataset made by co-simulating a power network and a communication network has been used. The decision trees are going to be compared in different settings by changing the division criteria, the dataset used to train them and the misclassification cost. After looking at their performance independently, the best way to combine them into a single algorithm is presented.

Trade-offs Between Accuracy and Health Outcomes in Algorithms for Home Blood Pressure Monitoring Devices

Objectives Home blood pressure monitoring (HBPM) is crucial for managing hypertension, but there is a potential trade-off between measurement accuracy and health/economic outcomes due to asymmetric costs associated with misclassifying an individual as having hypertension or not. We assessed whether adjustments to device readings that increased overall accuracy produced net health and economic benefits. Methods We analyzed data from N = 89 Alaska Native individuals who used 2 HBPM devices and a standard aneroid sphygmomanometer. We modeled changes in expected costs associated with individuals being misclassified as hypertensive or not under 3 different models of adjusting HBPM device readings. Results The gains in accuracy produced by adjusting HBPM readings decreased the overall rate of hypertension misclassification but increased the rate of false-negative readings. Adjusting readings led to a net increase in expected health and economic costs. Discussion Ignoring asymmetric costs of misclassification can escalate overall costs and worsen uncontrolled hypertension. Home blood pressure monitoring algorithms must be cautiously designed, considering both false negatives and positives. Greater transparency in HBPM algorithms is needed for effective coordination among manufacturers, clinicians, and patients.

Active Down-Sampling Method for Knn When Dealing with Imbalance Dataset

This study introduces Active Down-sampling (ADS), a novel approach combining downsampling with active learning to select informative samples from the majority class in imbalanced data scenarios, thereby enhancing machine learning model performance. Tested on three real-world datasets (BLOOD, Yeast, and Ecoli), ADS demonstrates superior classification accuracy over existing methods, efficiently balancing dataset representation while saving computational resources. It boosts accuracy across both minority and majority classes, optimizes resource use, and reduces misclassification costs. It emerges as a promising solution to the prevalent issue of data imbalance in machine learning, offering significant performance, resource, and cost advantages.

Predicting one-year readmission of adult patients with pneumonia using machine learning approaches: A case study in the Philippines

ABSTRACT Although several predictive models have been developed for patients with pneumonia who are at high risk of readmission, limited studies have been performed to assess pneumonia-related readmission within one year conducted in the Philippines. This study aimed to determine the factors associated with one-year readmission and to evaluate the suitability of predictive models for patients with pneumonia. A retrospective observational cohort study was conducted to evaluate pneumonia admissions in the partnered tertiary hospital from June 2017 to December 2018. Regularized logistic regression (RegLR), support vector machine (SVM), C5.0 decision tree, classification and regression tree (CART), and an artificial neural network (ANN) were used to develop the predictive models, which were evaluated using the area under the receiver operating characteristic curve (AUC). Of the 1,261 patients with pneumonia, 366 (29.02%) were hospitalized within one year after discharge. The factors associated with readmission included the initial laboratory test results and hospital utilization. The results showed that RegLR gave the best overall performance when using evaluation metrics as it resulted in the lowest misclassification cost. The findings of this study could be useful when developing strategies to reduce pneumonia readmission rates, enabling clinicians and hospitals to improve healthcare delivery for pneumonia patients.

Cost-sensitive continuous ensemble kernel learning for imbalanced data streams with concept drift

In stream learning, data continuously arrives over time, often at a very high rate. For imbalanced data streams with concept drift, it becomes essential to simultaneously address classification accuracy and time efficiency. However, existing algorithms currently either demand excessive computational time or exhibit relatively lower classification performance. To address these issues, in this paper, a cost-sensitive continuous ensemble kernel learning method (CCEKL) is proposed to deal with the imbalanced data streams with concept drift. Firstly, for two-class data streams, a novel misclassification cost is proposed to adapt to the changing imbalance rate. Secondly, based on the modified loss function, a continuous kernel learning method is applied to adapt to the changing data distribution. Furthermore, the two-class method is extended to multi-class classification to deal with multi-class dynamic imbalance problems. Finally, we use an ensemble approach to address the proposed algorithm sensitivity to the initial kernel width. To further improve efficiency, a parallel approach is applied to simultaneously train multi-classifiers with different initial kernel widths. Experimental results on 30 data streams reveal that, for binary classification, CCEKL achieves superior balanced accuracy in a shorter training time. For multi-class classification, CCEKL attains better balanced accuracy without requiring excessive training time, surpassing the most of the state-of-the-art baseline algorithms.

Detection of paralytic shellfish toxins by near-infrared spectroscopy based on a near-Bayesian SVM classifier with unequal misclassification costs.

Paralytic shellfish poisoning caused by human consumption of shellfish fed on toxic algae is a public health hazard. It is essential to implement shellfish monitoring programs to minimize the possibility of shellfish contaminated by paralytic shellfish toxins (PST) reaching the marketplace. This paper proposes a rapid detection method for PST in mussels using near-infrared spectroscopy (NIRS) technology. Spectral data in the wavelength range of 950-1700 nm for PST-contaminated and non-contaminated mussel samples were used to build the detection model. Near-Bayesian support vector machines (NBSVM) with unequal misclassification costs (u-NBSVM) were applied to solve a classification problem arising from the fact that the quantity of non-contaminated mussels was far less than that of PST-contaminated mussels in practice. The u-NBSVM model performed adequately on imbalanced datasets by combining unequal misclassification costs and decision boundary shifts. The detection performance of the u-NBSVM did not decline as the number of PST samples decreased due to adjustments to the misclassification costs. When the number of PST samples was 20, the G-mean and accuracy reached 0.9898 and 0.9944, respectively. Compared with the traditional support vector machines (SVMs) and the NBSVM, the u-NBSVM model achieved better detection performance. The results of this study indicate that NIRS technology combined with the u-NBSVM model can be used for rapid and non-destructive PST detection in mussels. © 2023 Society of Chemical Industry.