Cost-sensitive Classification Research Articles

A novel cost-sensitive three-way intuitionistic fuzzy large margin classifier

Three-way decision (3WD) has been widely applied in diverse fields in tackling uncertainty, particularly in classification domain. As a discriminative learning algorithm, Large Margin Distribution Machine (LDM) aims to maximize the inter-class margin by leveraging the marginal distribution of samples, which captures the underlying structure and achieves superior classification performance. However, existing LDMs still suffer from some inherent flaws, including: i) the neglect of fuzzy decision items, leading to the inadequate handling of uncertainty; ii) the utilization of a cost-insensitive mechanism, resulting in high misclassification costs; and iii) the disregard of sample credibility, contributing to the noise susceptibility. To address these limitations, we introduce the intuitionistic fuzzy (IF) and cost-sensitive 3WD (CS3WD), presenting an innovative model known as the CS3W-IFLMC model. The IF theory is employed to quantify sample confidence levels, augmenting noise suppression in our proposed model. Moreover, the integration of the CS3WD method effectively reduces the overall decision cost and further improves the handling of uncertainty in datasets. Consequently, the CS3W-IFLMC model demonstrates superior noise resilience and generalization performance. Our comparative experiments validate the efficacy of the proposed CS3W-IFLMC model in achieving robustness to noise while upholding a competitive classification performance.

Cost-sensitive ordinal classification methods to predict SARS-CoV-2 pneumonia severity.

To study the suitability of costsensitive ordinal artificial intelligence-machine learning (AIML) strategies in the prognosis of SARS-CoV-2 pneumonia severity. Observational, retrospective, longitudinal, cohort study in 4 hospitals in Spain. Information regarding demographic and clinical status was supplemented by socioeconomic data and air pollution exposures. We proposed AI-ML algorithms for ordinal classification via ordinal decomposition and for cost-sensitive learning via resampling techniques. For performance-based model selection, we defined a custom score including per-class sensitivities and asymmetric misprognosis costs. 260 distinct AI-ML models were evaluated via 10 repetitions of 5×5 nested cross-validation with hyperparameter tuning. Model selection was followed by the calibration of predicted probabilities. Final overall performance was compared against five well-established clinical severity scores and against a 'standard' (non-cost sensitive, non-ordinal) AI-ML baseline. In our best model, we also evaluated its explainability with respect to each of the input variables. The study enrolled n = 1548 patients: 712 experienced low, 238 medium, and 598 high clinical severity. d = 131 variables were collected, becoming d ' = 148 features after categorical encoding. Model selection resulted in our best-performing AI-ML pipeline having: a) no imputation of missing data, b) no feature selection (i.e. using the full set of d ' features), c) 'Ordered Partitions' ordinal decomposition, d) cost-based reimbalance, and e) a Histogram-based Gradient Boosting classifier. This best model (calibrated) obtained a median accuracy of 68.1% [67.3%, 68.8%] (95% confidence interval), a balanced accuracy of 57.0% [55.6%, 57.9%], and an overall area under the curve (AUC) 0.802 [0.795, 0.808]. In our dataset, it outperformed all five clinical severity scores and the 'standard' AI-ML baseline. We conducted an exhaustive exploration of AI-ML methods designed for both ordinal and cost-sensitive classification, motivated by a real-world application domain (clinical severity prognosis) in which these topics arise naturally. Our model with the best classification performance exploited successfully the ordering information of ground truth classes, coping with imbalance and asymmetric costs. However, these ordinal and cost-sensitive aspects are seldom explored in the literature.

Investigations of a controlled, decision tree based procedure for Selective Dry Cow Treatment in Bavarian dairy farms

Four parameters of a decision tree for Selective Dry Cow Treatment (SDCT), examined in a previous study, were analyzed regarding their efficacy in detecting cows for dry cow treatment (DCT, use of intramammary antimicrobials). This study set out to review wether all parameters (somatic cell count [SCC≥ 200 000 SC/ml 3 months' milk yield recordings prior dry off (DO)], clinical mastitis history during lactation [≥1 CM], culturing [14d prior DO, detection of major pathogens] and California-Mastitis-Test [CMT, > rate 1/+ at DO]) are necessary for accurate decision making, whether there are possible alternatives to replace culturing, and whether a simplified model could replace the decision tree. Records of 18 Bavarian dairy farms from June 2015 to August 2017 were processed. Data analysis was carried out by means of descriptive statistics, as well as employing a binary cost sensitive classification tree and logit-models. For statistical analyses the outcomes of the full 4-parameter decision tree were taken as ground truth. 848 drying off procedures in 739 dairy cows (CDO) were included. SCC and CMT selected 88.1%, in combination with CM 95.6% of the cows that received DCT (n=494). Without culturing, 22 (4.4%) with major pathogens (8x Staphylococcus [S.] aureus) infected CDO would have been misclassified as not needing DCT. The average of geometric mean SCC (within 100 d prior DO) for CDO with negative results in culturing was<100 000 SC/ml milk, 100 000-150 000 SC/ml for CDO infected with minor pathogens, and ≥ 150 000 SC/ml for CDO infected with major pathogens (excluding S.aureus). Using SCC during lactation (at least 1x > 200 000 SC/ml) and positive CMT to select CDO for DCT, contrary to the decision tree, 37 CDO (4.4%) would have been treated "incorrectly without" and 43 CDO (5.1%) "unnecessarily with" DCT. Modifications were identified, such as SCC<131 000 SC/ml within 100 d prior to DO for detecting CDO with no growth or minor pathogens in culturing. The best model for grading CDO for or against DCT (CDO without CM and SCC<200 000 SC/ml [last 3 months prior DO]) had metrics of AUC=0.74, Accuracy=0.778, balanced Accuracy=0.63, Sensitivity=0.92 and Specificity=0.33. Combining the decision tree's parameters SCC, CMT and CM renders suitable selection criteria under the conditions of this study. When omitting culturing, lower thresholds for SCC should be considered for each farm individually to select CDO for DCT. Nonetheless, the most accurate model could not replace the full decision tree.

Dynamic Treatment Strategy of Chinese Medicine for Metastatic Colorectal Cancer Based on Machine Learning Algorithm.

To establish the dynamic treatment strategy of Chinese medicine (CM) for metastatic colorectal cancer (mCRC) by machine learning algorithm, in order to provide a reference for the selection of CM treatment strategies for mCRC. From the outpatient cases of mCRC in the Department of Oncology at Xiyuan Hospital, China Academy of Chinese Medical Sciences, 197 cases that met the inclusion criteria were screened. According to different CM intervention strategies, the patients were divided into 3 groups: CM treatment alone, equal emphasis on Chinese and Western medicine treatment (CM combined with local treatment of tumors, oral chemotherapy, or targeted drugs), and CM assisted Western medicine treatment (CM combined with intravenous regimen of Western medicine). The survival time of patients undergoing CM intervention was taken as the final evaluation index. Factors affecting the choice of CM intervention scheme were screened as decision variables. The dynamic CM intervention and treatment strategy for mCRC was explored based on the cost-sensitive classification learning algorithm for survival (CSCLSurv). Patients' survival was estimated using the Kaplan-Meier method, and the survival time of patients who received the model-recommended treatment plan were compared with those who received actual treatment plan. Using the survival time of patients undergoing CM intervention as the evaluation index, a dynamic CM intervention therapy strategy for mCRC was established based on CSCLSurv. Different CM intervention strategies for mCRC can be selected according to dynamic decision variables, such as gender, age, Eastern Cooperative Oncology Group score, tumor site, metastatic site, genotyping, and the stage of Western medicine treatment at the patient's first visit. The median survival time of patients who received the model-recommended treatment plan was 35 months, while those who receive the actual treatment plan was 26.0 months (P=0.06). The dynamic treatment strategy of CM, based on CSCLSurv for mCRC, plays a certain role in providing clinical hints in CM. It can be further improved in future prospective studies with larger sample sizes.

An Application of Multinomial Mis-Classification Cost Matrix For A P- To – P Lending Credit Score

An emerging new form of online credit for lending, different from traditional sources of finance, such as banks and building societies, where lenders provide loans to borrowers directly is termed as P-P. Many of these credits are unsecured personal loans, thus credit score of loans is vital to regulate the default risk and improve profit for lenders and platforms. Standard two-fold classifiers may not be appropriate in this lending since there are multiple credit classes and misclassification costs vary largely across classes in the lending platforms. Cost Sensitive Classifiers have been studied extensively in this set of lending, but none of them have analyzed this issue from the perspective of multinomial classifications and measured the misclassification costs of different credit grades using actual losses and opportunity costs. The research intends to model credit score in p-p lending as a cost-sensitive multinomial classification problem. A misclassification cost matrix is proposed for credit scoring with a set of equations and models to estimate the costs. A replication study using a publicly available data is conducted to evaluate the performance and validate the usefulness of the proposed misclassification cost matrix with the help of an R statistical package developed to aid the application of the model. The outcomes showed that the cost-sensitive multinomial classifiers can significantly decrease the total cost, which is vital for the p-p survival and profitability.

Rethinking Cost-Sensitive Classification in Deep Learning via Adversarial Data Augmentation

Cost-sensitive classification is critical in applications where misclassification errors widely vary in cost. However, overparameterization poses fundamental challenges to the cost-sensitive modeling of deep neural networks (DNNs). The ability of a DNN to fully interpolate a training data set can render a DNN, evaluated purely on the training set, ineffective in distinguishing a cost-sensitive solution from its overall accuracy maximization counterpart. This necessitates rethinking cost-sensitive classification in DNNs. To address this challenge, this paper proposes a cost-sensitive adversarial data augmentation (CSADA) framework to make overparameterized models cost sensitive. The overarching idea is to generate targeted adversarial examples that push the decision boundary in cost-aware directions. These targeted adversarial samples are generated by maximizing the probability of critical misclassifications and used to train a model with more conservative decisions on costly pairs. Experiments on well-known data sets and a pharmacy medication image (PMI) data set, made publicly available, show that our method can effectively minimize the overall cost and reduce critical errors while achieving comparable overall accuracy. Funding: Research reported in this publication was supported by the National Library of medicine of the National Institutes of Health in the United States under award number R01LM013624. Data Ethics &amp; Reproducibility Note: This paper abides by data ethics requirements. Data used are publicly available online at https://deepblue.lib.umich.edu/data/concern/data_sets/6d56zw997 . Codes to replicate the results of this paper are available at https://doi.org/10.24433/CO.2139841.v1 .

Decision-Making on the Diagnosis of Oncological Diseases Using Cost-Sensitive SVM Classifiers Based on Datasets with a Variety of Features of Different Natures

This paper discusses the problem of detecting cancer using such biomarkers as blood protein markers. The purpose of this research is to propose an approach for making decisions in the diagnosis of cancer through the creation of cost-sensitive SVM classifiers on the basis of datasets with a variety of features of different nature. Such datasets may include compositions of known features corresponding to blood protein markers and new features constructed using methods for calculating entropy and fractal dimensions, as well as using the UMAP algorithm. Based on these datasets, multiclass SVM classifiers were developed. They use cost-sensitive learning principles to overcome the class imbalance problem, which is typical for medical datasets. When implementing the UMAP algorithm, various variants of the loss function were considered. This was performed in order to select those that provide the formation of such new features that ultimately allow us to develop the best cost-sensitive SVM classifiers in terms of maximizing the mean value of the metric MacroF1−score. The experimental results proved the possibility of applying the UMAP algorithm, approximate entropy and, in addition, Higuchi and Katz fractal dimensions to construct new features using blood protein markers. It turned out that when working with the UMAP algorithm, the most promising is the application of a loss function on the basis of fuzzy cross-entropy, and the least promising is the application of a loss function on the basis of intuitionistic fuzzy cross-entropy. Augmentation of the original dataset with either features on the basis of the UMAP algorithm, features on the basis of the UMAP algorithm and approximate entropy, or features on the basis of approximate entropy provided the creation of the three best cost-sensitive SVM classifiers with mean values of the metric MacroF1−score increased by 5.359%, 5.245% and 4.675%, respectively, compared to the mean values of this metric in the case when only the original dataset was utilized for creating the base SVM classifier (without performing any manipulations to overcome the class imbalance problem, and also without introducing new features).

Examining different cost ratio frameworks for decision rule machine learning algorithms in diagnostic application.

Artificial Intelligence (AI) plays a pivotal role in the diagnosis of health conditions ranging from general well-being to critical health issues. In the realm of health diagnostics, an often overlooked but critical aspect is the consideration of cost-sensitive learning, a facet that this study prioritizes over the non-invasive nature of the diagnostic process whereas the other standard metrics such as accuracy and sensitivity reflect weakness in error profile. This research aims to investigate the total cost of misclassification (Total Cost) by decision rule Machine Learning (ML) algorithms implemented in Java platforms such as DecisionTable, JRip, OneR, and PART. An augmented dataset with conjunctiva images along candidates' demographic and anthropometric features under supervised learning is considered with a specific emphasis on cost-sensitive classification. The opted decision rule classifiers use the text features, additionally the image feature 'a* value of CIELAB color space' extracted from the conjunctiva digital images as input attributes. The pre-processing consists of amalgamating text and image features on a uniform scale, normalizing. Then the 10-fold cross-validation enables the classification of samples into two categories: the presence or absence of the anemia. This study utilizes the Cost Ratio (ρ) extracted from the cost matrix to meticulously monitor the Total Cost in four different cost ratio methodologies namely Uniform (U), Uniform Inverted (UI), Non-Uniform (NU), and Non-Uniform Inverted (NUI). It has been established that the PART classifier stands out as the top performer in this binary classification task, yielding the lowest mean total cost of 629.9 compared to other selected classifiers. Moreover, it demonstrates a comparatively lower standard deviation 335.9, and lower total cost across all four different cost ratio methodologies. The ranking of algorithm performance goes as follows: PART, JRIP, DecisionTable, and OneR. The significance of adopting a cost-sensitive learning approach is emphasized showing the PART classifier's consistent performance within the proposed framework for learning the anemia dataset. This emphasis on cost-sensitive learning not only enhances the recommendations in diagnosis but also holds the potential for substantial cost savings and makes it a noteworthy focal point in the advancement of AI-driven health care.

Research on Intrusion Detection Based on an Enhanced Random Forest Algorithm

To address the challenges posed by high data dimensionality and class imbalance during intrusion detection, which result in increased computational complexity, resource consumption, and reduced classification accuracy, this paper presents an intrusion-detection algorithm based on an improved Random Forest approach. The algorithm employs the Bald Eagle Search (BES) optimization technique to fine-tune the Kernel Principal Component Analysis (KPCA) algorithm, enabling optimized dimensionality reduction. The processed data are then fed into a cost-sensitive Random Forest classifier for training, with subsequent model validation conducted on the reduced-dimension data. Experimental results demonstrate that compared to traditional Random Forest algorithms, the proposed method reduces the training time by 11.32 s and achieves a 5.59% increase in classification accuracy, an 11.7% improvement in specificity, and a 0.0558 increase in the G-mean value. These findings underscore the promising application potential and performance of this approach in the field of network intrusion detection.

An Explainable AI Approach for Breast Cancer Metastasis Prediction Based on Clinicopathological Data.

Breast Cancer is the most prevalent cancer and the first cause of cancer deaths among women worldwide. In 90% of the cases, mortality is related to distant metastasis. Computer-aided prognosis systems using machine learning models have been widely used to predict breast cancer metastasis. Despite that, these systems still face several challenges. First, the models are generally biased toward the majority class due to datasets unbalance. Second, their increased complexity is associated with decreased interpretability which causes clinicians to distrust their prognosis. To tackle these issues, we have proposed an explainable approach for predicting breast cancer metastasis using clinicopathological data. Our approach is based on cost-sensitive CatBoost classifier and utilises LIME explainer to provide patient-level explanations. We used a public dataset of 716 breast cancer patients to assess our approach. The results demonstrate the superiority of cost-sensitive CatBoost in precision (76.5%), recall (79.5%), and f1-score (77%) over classical and boosting models. The LIME explainer was used to quantify the impact of patient and treatment characteristics on breast cancer metastasis, revealing that they have different impacts ranging from high impact like the non-use of adjuvant chemotherapy, and moderate impact including carcinoma with medullary features histological type, to low impact like oral contraception use. The code is available at https://github.com/IkramMaouche/CS-CatBoost Conclusion: Our approach serves as a first step toward introducing more efficient and explainable computer-aided prognosis systems for breast cancer metastasis prediction. This approach could help clinicians understand the factors behind metastasis and assist them in proposing more patient-specific therapeutic decisions.

Reinforcement learning-based cost-sensitive classifier for imbalanced fault classification

Reinforcement learning-based cost-sensitive classifier for imbalanced fault classification

Comparing cost sensitive classifiers by the false-positive to false- negative ratio in diagnostic studies

Comparing cost sensitive classifiers by the false-positive to false- negative ratio in diagnostic studies

Cost-Sensitive Online Adaptive Kernel Learning for Large-Scale Imbalanced Classification

Imbalanced classification is a challenging task in the fields of machine learning, data mining and pattern recognition. Cost-sensitive online algorithms are very important methods for large-scale imbalanced classification problems. At present, most of the cost-sensitive classification algorithms focus on the accuracy of the minority class and ignore the accuracy of the majority class. In order to better balance the accuracy between the minority class and the majority class, in this paper, a misclassification cost is presented to ensure that the cost-sensitive online algorithm can better deal with the imbalanced classification problems without signifificantly reducing the accuracy of the majority class. Based on the proposed misclassification cost, a novel cost-sensitive online adaptive kernel learning algorithm is proposed to boost the adaptability of kernel function when data arrives one by one. According to the essential characteristics of the imbalanced binary classification, a cost-sensitive online adaptive kernel learning algorithm is given to handle the large-scale imbalanced multi-class classification problems. Theoretical analysis of the proposed algorithms are provided. Extensive experiments demonstrate that compared with the state-of-the-art imbalanced classification algorithms, the proposed algorithms can significantly improve the classification performances on most of the large-scale imbalanced data sets.

Self-optimised cost-sensitive classifiers for early field failure prediction in storage systems

Data storage systems such as disk arrays go through rigorous testing in the production phase, however, a few of these DAs fail in the field and are returned back to the manufacturer. Although the failure appears in relatively a small percentage of the manufactured DAs, it results in a significant loss of data, time and money. This paper is motivated by the hypothesis that many of these failures could be predicted at the testing stage through data mining and machine learning. Field failure is modelled as a classification problem, however, as in many real-world problems, the problem suffers from significant class imbalance. Several approaches have been proposed that attempt to improve the performance of imbalanced classification by either modifying the dataset (resampling), or assigning classification costs to the classes’ cost matrix. These methods have been shown to improve performance, but they come with many parameters that need to be set, something that usually requires a lengthy exhaustive search, especially on problems with several classes. This paper presents a new scalable genetic algorithm approach for automating the design of the cost matrix CM along with the algorithm parameters. The proposed algorithms are tested on a real-world manufacturing dataset from Seagate disk arrays; the target is to predict from the devices’ testing data those that are likely to fail in the field. To demonstrate its performance, the proposed approach evaluated on a number of standard datasets and compared with other state-of-the-art methods.

Adversarial Learning With Cost-Sensitive Classes.

It is necessary to improve the performance of some special classes or to particularly protect them from attacks in adversarial learning. This article proposes a framework combining cost-sensitive classification and adversarial learning together to train a model that can distinguish between the protected and unprotected classes, such that the protected classes are less vulnerable to adversarial examples. We find in this framework an interesting phenomenon during the training of deep neural networks, called the Min-Max property, that is, the absolute values of most parameters in the convolutional layer approach 0 while the absolute values of a few parameters are significantly larger, becoming bigger. Based on this Min-Max property which is formulated and analyzed in a view of random distribution, we further build a new defense model against adversarial examples for adversarial robustness improvement. An advantage of the built model is that it performs better than the standard one and can combine with adversarial training to achieve improved performance. It is experimentally confirmed that, regarding the average accuracy of all classes, our model is almost as same as the existing models when an attack does not occur and is better than the existing models when an attack occurs. Specifically, regarding the accuracy of protected classes, the proposed model is much better than the existing models when an attack occurs.

Generalized Transfer Extreme Learning Machine for Unsupervised Cross-Domain Fault Diagnosis With Small and Imbalanced Samples

In most engineering scenarios, the collected datasets typically have small and imbalanced characteristics, and usually possess different data distributions between variable working conditions. Thus, several existing diagnosis models trained by small and imbalanced samples under one working condition may suffer from low accuracy for fault data under another working condition. To effectively alleviate such problem, a novel domain adaptation method, referred to as generalized transfer extreme learning machine (G-TELM), is proposed in this study. First, in a random feature space, marginal and conditional distribution discrepancy between source and target domains is minimized based on projected maximum mean discrepancy. Second, in order to give varied attention to different faulty samples, a cost-sensitive source domain classifier is utilized to reduce the difference between types of misclassification loss for small and imbalanced source data. Third, misclassification loss of target data is used to minimize the difference between the network’s outputs and prediction labels of target data. Last, L2-norm regularization is leveraged to constrain network output weights, and they are iteratively updated to optimize the transferability of model G-TELM. Experimental results of 168 transfer tasks with varying difficulty verify that the proposed model G-TELM can achieve an accurate diagnosis and outperform several state-of-art domain adaptation methods.

Cost-sensitive classification algorithm combining the Bayesian algorithm and quantum decision tree

This study highlights the drawbacks of current quantum classifiers that limit their efficiency and data processing capabilities in big data environments. The paper proposes a global decision tree paradigm to address these issues, focusing on designing a complete quantum decision tree classification algorithm that is accurate and efficient while also considering classification costs. The proposed method integrates the Bayesian algorithm and the quantum decision tree classification algorithm to handle incremental data. The proposed approach generates a suitable decision tree dynamically based on data objects and cost constraints. To handle incremental data, the Bayesian algorithm and quantum decision tree classification algorithm are integrated, and kernel functions obtained from quantum kernel estimation are added to a linear quantum support vector machine to construct a decision tree classifier using decision directed acyclic networks of quantum support vector machine nodes (QKE). The experimental findings demonstrate the effectiveness and adaptability of the suggested quantum classification technique. In terms of classification accuracy, speed, and practical application impact, the proposed classification approach outperforms the competition, with an accuracy difference from conventional classification algorithms being less than 1%. With improved accuracy and reduced expense as the incremental data increases, the efficiency of the suggested algorithm for incremental data classification is comparable to previous quantum classification algorithms. The proposed global decision tree paradigm addresses the critical issues that need to be resolved by quantum classification methods, such as the inability to process incremental data and the failure to take the cost of categorization into account. By integrating the Bayesian algorithm and the quantum decision tree classification algorithm and using QKE, the proposed method achieves high accuracy and efficiency while maintaining high performance when processing incremental sequences and considering classification costs. Overall, the theoretical and experimental findings demonstrate the effectiveness of the suggested quantum classification technique, which offers a promising solution for handling big data classification tasks that require high accuracy and efficiency.

Assessment and classification of grid stability with cost-sensitive stacked ensemble classifier

Smart Grid is an intelligent power grid with a bidirectional flow of electricity and information, that applies intelligent techniques to operate the grid autonomously near the stability limit. An intelligent technique is developed to identify and predict the abnormalities due to changes in customer behaviour and the unexpected disruption in the grid. A cost-sensitive stacked ensemble classifier (CS-SEC) is proposed for predicting the operations in smart grid that combines four cost-sensitive base classifiers, namely Extreme gradient boosting, Naive Bayes, Nu-support vector machine and Random forest at level-1 and the support vector machine as the meta classifier in level-2. The meta classifier uses the probability of prediction of the first-level classifiers with stratified 5-fold cross-validation to predict the decentralized smart grid stability. The proposed stacked ensemble classifier achieved an accuracy of 98.6% with specificity, recall and precision of 98.34%, 99.0% and 99.06%, respectively. Extensive experimental evaluation and results show that the proposed CS-SEC provides an accurate prediction of grid stability compared with other state-of-the-art models. The results reveal the robustness and competency of the proposed CS-SECs with optimized parameters.

Classifier calibration: a survey on how to assess and improve predicted class probabilities

This paper provides both an introduction to and a detailed overview of the principles and practice of classifier calibration. A well-calibrated classifier correctly quantifies the level of uncertainty or confidence associated with its instance-wise predictions. This is essential for critical applications, optimal decision making, cost-sensitive classification, and for some types of context change. Calibration research has a rich history which predates the birth of machine learning as an academic field by decades. However, a recent increase in the interest on calibration has led to new methods and the extension from binary to the multiclass setting. The space of options and issues to consider is large, and navigating it requires the right set of concepts and tools. We provide both introductory material and up-to-date technical details of the main concepts and methods, including proper scoring rules and other evaluation metrics, visualisation approaches, a comprehensive account of post-hoc calibration methods for binary and multiclass classification, and several advanced topics.

75P Electronic patient-reported outcomes and machine learning in predicting unplanned visits and hospitalisation rates in cancer patients undergoing systemic therapy

Electronic patient-reported outcomes (ePRO) have shown the potential to improve clinical practice for cancer patients. The collection of ePRO via smartphone apps is becoming more wide-spread. Such data, comprising of symptom and diary entries, could conceivably be utilised to predict the occurrence of unplanned hospitalisation events via machine learning algorithms. 226 cancer patients (mainly breast cancer) using the consilium care ePRO app under systemic therapy. A diary entry is associated with an “unplanned visit” flag if such a visit or a hospitalization of the patient occurred on or within 3 days. We assume that on these days, an alarm raised to the patient or her/his physician would have been helpful. Out of 16,670 entries, only 166 (1%) were flagged. To implement an early warning system, we trained a rule-based ML algorithm to predict critical situations, which we made more sensitive to the few flagged entries by applying cost-sensitive classification, assigning a 10-fold higher cost to missing a critical situation as compared to a false alarm. Rules are using are based on the strengths of symptoms captured by the patients, their perceived wellbeing, drugs they were taking, as well as keywords derived from free-text notes of patients and from their diagnosis; e.g. one of our rules suggests that an unplanned visit is likely to occur if a patient reports pain on a level of 38 or more and a wellbeing of 60 or less (both out of 100) and uses the word “today” in his/her notes of the given day. We intentionally used algorithms that are potentially inferior in prediction quality to modern deep learning approaches, but have the advantage of being not only human-interpretable, but even directly “human-modifiable” so that we can incorporate medical expertise into the machine-learned model at a later stage. We discovered a set of 55 rules which, as a whole, were able to correctly predict 47 of the 166 critical situations (28.3% recall), while raising 267 false alarms (15.2% precision). Preliminary analysis of this study suggests that ML based prediction models trained on ePRO data can predict unplanned hospitalisation events within cancer patients.