Classification Performance Measures Research Articles

Doubly Robust Augmented Model Accuracy Transfer Inference with High Dimensional Features

Transfer learning is crucial for training models that generalize to unlabeled target populations using labeled source data, especially in real-world studies where label scarcity and covariate shift are common. While most research focuses on model estimation, there is limited literature on transfer inference for model accuracy despite its importance. We introduce a novel Doubly Robust Augmented Model Accuracy Transfer Inferen Ce (DRAMATIC) method for point and interval estimation of commonly used classification performance measures in an unlabeled target population with labeled source data. DRAMATIC derives and evaluates a potentially misspecified risk model for a binary response, leveraging high-dimensional adjustment features from both source and target data. It builds on an imputation model for the response mean and a density ratio model to characterize distributional shifts. The method constructs doubly robust estimators that are valid when either model is correctly specified and certain sparsity assumptions hold. Simulations show negligible bias in point estimation and satisfactory empirical coverage levels in confidence intervals. The utility of DRAMATIC is illustrated by transferring a genetic risk prediction model and its accuracy evaluation for type II diabetes across two patient cohorts in Mass General Brigham (MGB). Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

SENTIMENT/CYBER BULLYING DETECTION USING PYTHON & MACHINE LEARNING

In the modern era, the usage of the internet has increased tremendously which in turn has led to the evolution of large amount of data. Cyber world has its own pros and cons. One of the alarming situations in web 4.0 is cyber bullying a type of cyber-crime. When bullying occurs online with the aid of technology it is known as cyber bullying. This research paper has surveyed the work done by 30 different researchers on cyber bullying, and elaborated on different methodologies adopted by them for the detection of bullying. Three types of features namely textual, behavioral and demographic features are extracted from the dataset as compared to earlier study over the same dataset where only textual features were considered. Textual features include certain bullying words that if exists within the text may lead to a true outcome for cyber bullying. Personality trait features are extracted for the user if it is involved once in bullying may bully in future too. While demographic features extracted from dataset include age, gender and location. The system is evaluated through different performance measures for both classifiers used the performance of the Support Vector Machine classifier is found better than the Bernoulli NB with overall 87.14 Accuracy. Keywords: Cyberbully, Support Vector Machine, Textual, classifier.

Explainable AI: Machine Learning Interpretation in Blackcurrant Powders.

Recently, explainability in machine and deep learning has become an important area in the field of research as well as interest, both due to the increasing use of artificial intelligence (AI) methods and understanding of the decisions made by models. The explainability of artificial intelligence (XAI) is due to the increasing consciousness in, among other things, data mining, error elimination, and learning performance by various AI algorithms. Moreover, XAI will allow the decisions made by models in problems to be more transparent as well as effective. In this study, models from the 'glass box' group of Decision Tree, among others, and the 'black box' group of Random Forest, among others, were proposed to understand the identification of selected types of currant powders. The learning process of these models was carried out to determine accuracy indicators such as accuracy, precision, recall, and F1-score. It was visualized using Local Interpretable Model Agnostic Explanations (LIMEs) to predict the effectiveness of identifying specific types of blackcurrant powders based on texture descriptors such as entropy, contrast, correlation, dissimilarity, and homogeneity. Bagging (Bagging_100), Decision Tree (DT0), and Random Forest (RF7_gini) proved to be the most effective models in the framework of currant powder interpretability. The measures of classifier performance in terms of accuracy, precision, recall, and F1-score for Bagging_100, respectively, reached values of approximately 0.979. In comparison, DT0 reached values of 0.968, 0.972, 0.968, and 0.969, and RF7_gini reached values of 0.963, 0.964, 0.963, and 0.963. These models achieved classifier performance measures of greater than 96%. In the future, XAI using agnostic models can be an additional important tool to help analyze data, including food products, even online.

Anomaly detection using deep convolutional generative adversarial networks in the internet of things

Advanced 5 G and 6 G technologies have accelerated the adoption of the Internet of Things (IoT) and are a priority in providing support for high-speed communication and fast data analysis. One of IoT networks benefits is automated networking, which unfortunately increases the risk of security, integrity, and privacy breaches. Therefore, in this paper, we propose a weighted stacked ensemble model combining deep convolutional generative adversarial and bidirectional long short-term memory networks. The proposed model has been regularized, and hyperparameter tuning has been performed. The tuned model is then evaluated on four publicly available current IoT datasets. The proposed model exhibits significant improvement in standard performance measures for both binary and multiclass classification. Generalization error has been reduced by a rate of 0.005% and to overcome the issue of overfitting, a L2 regularization technique has been deployed. The overall Accuracy of the model on various datasets is 99.99% for BOT-IoT, 99.08% for IoT23, 99.82% for UNSWNB15, and 99.96% for ToN_IoT, respectively, alongside improvements in Precision, Recall, and F1-score.

Tree-Based Ensemble Models, Algorithms and Performance Measures for Classification

Tree-Based Ensemble Models, Algorithms and Performance Measures for Classification

Recent Advancement in Prediction and Analyzation of Brain Tumour using the Artificial Intelligence Method

Brain tumour identification and categorization are critical for diagnosis and treatment. This work uses preprocessing and classification algorithms to discover and categories brain tumours. Gaussian smoothing reduces noise and improves image quality, Genetic Algorithms select and optimize features, Deep Learning-Based Segmentation accurately segments tumours, Local Binary Patterns (LBP) extract texture features, and K-Nearest Neighbors classify tumours. Gaussian smoothing reduces noise and improves brain imaging data. Genetic Algorithms extract the most relevant and discriminative features from preprocessed photos. A Deep Learning-Based Segmentation model accurately segments brain tumour regions using these features. After segmentation, Local Binary Patterns (LBP) extract tumour texture features. The K-Nearest Neighbors (KNN) method classifies tumours using these texture features to capture tumour spatial patterns. Our suggested brain tumour detection and classification method combines several techniques to increase accuracy and reliability. Gaussian smoothing and LBP feature extraction improve feature discrimination. Deep Learning-Based Segmentation and the KNN classifier ensure precise tumour location and robust classification. The proposed method will be tested on brain scans including tumour areas. Classification performance measures include accuracy, sensitivity, specificity, and AUC. This work will improve brain tumour detection and classification methods for more accurate diagnoses and treatment planning. The primary goal of the work is to enhance brain tumour identification and categorization using pre processing, classification algorithms, and advanced techniques, ultimately improving diagnosis and treatment outcomes.

Opt-RNN-DBFSVM: Optimal recurrent neural network density based fuzzy support vector machine

Two major problems are encountered when using fuzzy SVM: (a) the number of local minima increases exponentially with the number of samples and (b) the quantity of required computer storage, required for a regular quadratic programming solver, increases by an exponential magnitude as the problem size expands. The Kernel-Adatron family of algorithms gaining attention lately which has allowed to handle very large classification and regression problems. However, these methods treat different types of samples (Noise, border, and core) with the same manner, which causes searches in unpromising areas and increases the number of iterations. In this work, we introduce a hybrid method to overcome these shortcoming, namely Optimal Recurrent Neural Network Density Based fuzzy Support Vector Machine (Opt-RNN-DBFSVM). This method consists of four steps: (a) characterization of different samples, (b) elimination of samples with a low probability of being a support vector, (c) construction of an appropriate recurrent neural network based on an original energy function, and (d) solution of the system of differential equations, managing the dynamics of the RNN, using the Euler–Cauchy method involving an optimal time step. Thanks to its recurrent architecture, the RNN remembers the regions explored during the search process. We demonstrated that RNN-FSVM converges to feasible support vectors and Opt-RNN-DBFSVM has a very low time complexity compared to RNN-FSVM with constant time step, and KAs-FSVM. Several experiments were performed on academic data sets. We used several classification performance measures to compare Opt-RNN-DBFSVM to different classification methods and the results obtained show the good performance of the proposed method.

Stacked deep model‐based classification of the multiclass brain hemorrhages in CT scans

AbstractIntracranial hemorrhage (ICH) brain hemorrhages are life‐threatening medical conditions that require a prompt and accurate diagnosis to ensure timely intervention and treatment. Computed tomography (CT) scans play a vital role in diagnosing brain hemorrhages due to their ability to provide detailed cross‐sectional images of the brain. However, the manual interpretation of CT scans is time‐consuming and subjects to human error. To address these challenges, this study proposes a stacked deep model‐based classification approach for the automated identification and differentiation of multiclass brain hemorrhages in CT scans. The stacked deep model architecture consists of multiple layers of neural networks, each designed to learn specific levels of features, leading to enhanced representation learning and improved classification performance. Later, the stacked models are connected with the weighted approach that helps to get the ensemble models in the proper order based on their power of predicting correct samples. To build the stacked deep model, a large dataset of 7,52 000 annotated CT scans is utilized to train the model and optimize its parameters. Furthermore, transfer learning is employed to leverage pretrained models for feature extraction, which aids in handling limited data scenarios. Experimental results demonstrate that the stacked deep model achieves superior accuracy (98.56%), and other classification performance measures such as sensitivity (95.89%), specificity (99.58%), AUC‐ROC (98.47%), and weighted log loss (0.04967) as compared to conventional methods. The automated classification of multiclass brain hemorrhages in CT scans using the proposed model not only significantly reduces the time required for diagnosis but also enhances the reliability and consistency of results. This model has the potential to assist medical professionals in making more informed decisions and improve patient outcomes by enabling rapid and accurate diagnosis of brain hemorrhages.

OPT-RNN-DBSVM: OPTimal Recurrent Neural Network and Density-Based Support Vector Machine

When implementing SVMs, two major problems are encountered: (a) the number of local minima of dual-SVM increases exponentially with the number of samples and (b) the computer storage memory required for a regular quadratic programming solver increases exponentially as the problem size expands. The Kernel-Adatron family of algorithms, gaining attention recently, has allowed us to handle very large classification and regression problems. However, these methods treat different types of samples (i.e., noise, border, and core) in the same manner, which makes these algorithms search in unpromising areas and increases the number of iterations as well. This paper introduces a hybrid method to overcome such shortcomings, called the Optimal Recurrent Neural Network and Density-Based Support Vector Machine (Opt-RNN-DBSVM). This method consists of four steps: (a) the characterization of different samples, (b) the elimination of samples with a low probability of being a support vector, (c) the construction of an appropriate recurrent neural network to solve the dual-DBSVM based on an original energy function, and (d) finding the solution to the system of differential equations that govern the dynamics of the RNN, using the Euler–Cauchy method involving an optimal time step. Density-based preprocessing reduces the number of local minima in the dual-SVM. The RNN’s recurring architecture avoids the need to explore recently visited areas. With the optimal time step, the search moves from the current vectors to the best neighboring support vectors. It is demonstrated that RNN-SVM converges to feasible support vectors and Opt-RNN-DBSVM has very low time complexity compared to the RNN-SVM with a constant time step and the Kernel-Adatron algorithm–SVM. Several classification performance measures are used to compare Opt-RNN-DBSVM with different classification methods and the results obtained show the good performance of the proposed method.

Kernel-Based Regularized EEGNet Using Centered Alignment and Gaussian Connectivity for Motor Imagery Discrimination

Brain–computer interfaces (BCIs) from electroencephalography (EEG) provide a practical approach to support human–technology interaction. In particular, motor imagery (MI) is a widely used BCI paradigm that guides the mental trial of motor tasks without physical movement. Here, we present a deep learning methodology, named kernel-based regularized EEGNet (KREEGNet), leveled on centered kernel alignment and Gaussian functional connectivity, explicitly designed for EEG-based MI classification. The approach proactively tackles the challenge of intrasubject variability brought on by noisy EEG records and the lack of spatial interpretability within end-to-end frameworks applied for MI classification. KREEGNet is a refinement of the widely accepted EEGNet architecture, featuring an additional kernel-based layer for regularized Gaussian functional connectivity estimation based on CKA. The superiority of KREEGNet is evidenced by our experimental results from binary and multiclass MI classification databases, outperforming the baseline EEGNet and other state-of-the-art methods. Further exploration of our model’s interpretability is conducted at individual and group levels, utilizing classification performance measures and pruned functional connectivities. Our approach is a suitable alternative for interpretable end-to-end EEG-BCI based on deep learning.

Interpretable Computer Vision to Detect and Classify Structural Laryngeal Lesions in Digital Flexible Laryngoscopic Images.

To localize structural laryngeal lesions within digital flexible laryngoscopic images and to classify them as benign or suspicious for malignancy using state-of-the-art computer vision detection models. Cross-sectional diagnostic study SETTING: Tertiary care voice clinic METHODS: Digital stroboscopic videos, demographic and clinical data were collected from patients evaluated for a structural laryngeal lesion. Laryngoscopic images were extracted from videos and manually labeled with bounding boxes encompassing the lesion. Four detection models were employed to simultaneously localize and classify structural laryngeal lesions in laryngoscopic images. Classification accuracy, intersection over union (IoU) and mean average precision (mAP) were evaluated as measures of classification, localization, and overall performance, respectively. In total, 8,172 images from 147 patients were included in the laryngeal image dataset. Classification accuracy was 88.5 for individual laryngeal images and increased to 92.0 when all images belonging to the same sequence (video) were considered. Mean average precision across all four detection models was 50.1 using an IoU threshold of 0.5 to determine successful localization. Results of this study showed that deep neural network-based detection models trained using a labeled dataset of digital laryngeal images have the potential to classify structural laryngeal lesions as benign or suspicious for malignancy and to localize them within an image. This approach provides valuable insight into which part of the image was used by the model to determine a diagnosis, allowing clinicians to independently evaluate models' predictions.

Copula modeling for the estimation of measures of marker classification and predictiveness performance with survival outcomes.

The discriminative and predictive power of a continuous-valued marker for survival outcomes can be summarized using the receiver operating characteristic and predictiveness curves, respectively. In this paper, fully parametric and semi-parametric copula-based constructions of the joint model of the marker and the survival time are developed for characterizing, plotting, and analyzing both curves along with other underlying performance measures. The formulations require a copula function, a parametric specification for the margin of the marker, and either a parametric distribution or a non-parametric estimator for the margin of the time to event, to respectively characterize the fully parametric and semi-parametric joint models. Estimation is carried out using maximum likelihood and a two-stage procedure for the parametric and semi-parametric models, respectively. Resampling-based methods are used for computing standard errors and confidence bounds for the various parameters, curves, and associated measures. Graphical inspection of residuals from each conditional distribution is employed as a guide for choosing a copula from a set of candidates. The performance of the estimators of various classification and predictiveness measures is assessed in simulation studies, assuming different copula and censoring scenarios. The methods are illustrated with the analysis of two markers using the familiar primary biliary cirrhosis data set.

Effect of Missing Data Types and Imputation Methods on Supervised Classifiers: An Evaluation Study

Data completeness is one of the most common challenges that hinder the performance of data analytics platforms. Different studies have assessed the effect of missing values on different classification models based on a single evaluation metric, namely, accuracy. However, accuracy on its own is a misleading measure of classifier performance because it does not consider unbalanced datasets. This paper presents an experimental study that assesses the effect of incomplete datasets on the performance of five classification models. The analysis was conducted with different ratios of missing values in six datasets that vary in size, type, and balance. Moreover, for unbiased analysis, the performance of the classifiers was measured using three different metrics, namely, the Matthews correlation coefficient (MCC), the F1-score, and accuracy. The results show that the sensitivity of the supervised classifiers to missing data differs according to a set of factors. The most significant factor is the missing data pattern and ratio, followed by the imputation method, and then the type, size, and balance of the dataset. The sensitivity of the classifiers when data are missing due to the Missing Completely At Random (MCAR) pattern is less than their sensitivity when data are missing due to the Missing Not At Random (MNAR) pattern. Furthermore, using the MCC as an evaluation measure better reflects the variation in the sensitivity of the classifiers to the missing data.

Iterative Reflect Perceptual Sammon and Machine Learning-Based Bagging Classification for Efficient Tumor Detection

A tumor is an abnormal development of cells in the human body. A tumor develops when cells divide without any control. Tumors change their size from a small to large lump. Tumors appear anywhere in the body. The early stage of diagnosis is an essential one in disease treatment. Many researchers carried out their research on different tumor detection methods. However, the tumor detection accuracy level was not improved and tumor detection time consumption not minimized. In order to address these problems, an Iterative Reflect Perceptual Sammon Bagging Classification (IRPS-BAC) Method is introduced. The aim is to accurately detect brain tumors as early as possible and make the method suitable for real-time applications. The IRPS-BAC Method comprises two processes, namely, feature selection and classification using the iterative reflect perceptual sammon feature selection process and bagging classification process. In the IRPS-BAC Method, an input of medical data are gathered from the Epileptic Seizure Recognition Data Set and Cervical Cancer Risk Classification database. After that, iterative reflect perceptual sammon feature selection process is carried out to select the relevant features. Iterative reflect perceptual divergence computes the variation between two features. After that, sammon mapping projects the similar and dissimilar features into feature space. By this manner, the relevant features get selected using the IRPS-BAC Method. With the help of selected relevant features, bagging classification process is carried out. In bagging classification process, internal node processes the selected features and leaf node to make the tumor decision as normal or cancerous one based on information gain. This, in turn, helps to reduce the time complexity and error rate. The performance of the proposed IRPS-BAC Method is determined by two benchmark datasets through comparing the parameter such as tumor detection time, tumor detection accuracy and error rate with the existing approaches. In the Epileptic Seizure Recognition Data Set, the proposed IRPS-BAC Method improves tumor detection accuracy by 16%, with minimum time period and the error rate of 41 ms and 58% for tumor detection as compared to existing methods. By using Cervical Cancer Risk Classification, the proposed IRPS-BAC Method exhibited higher classification performance measures, including accuracy (14%), time (46 ms), and error rate (61%), than the current conventional approaches.

Machine learning analysis of breast ultrasound to classify triple negative and HER2+ breast cancer subtypes.

Early diagnosis of triple-negative (TN) and human epidermal growth factor receptor2 positive (HER2+) breast cancer is important due to its increased risk of micrometastatic spread necessitating early treatment and for guiding targeted therapies. This study aimed to evaluate the diagnostic performance of machine learning (ML) classification of newly diagnosed breast masses into TN versus non-TN (NTN) and HER2+ versus HER2 negative (HER2-) breast cancer, using radiomic features extracted from grayscale ultrasound (US) b-mode images. A retrospective chart review identified 88 female patients who underwent diagnostic breast US imaging, had confirmation of invasive malignancy on pathology and receptor status determined on immunohistochemistry available. The patients were classified as TN, NTN, HER2+ or HER2- for ground-truth labelling. For image analysis, breast masses were manually segmented by a breast radiologist. Radiomic features were extracted per image and used for predictive modelling. Supervised ML classifiers included: logistic regression, k-nearest neighbour, and Naïve Bayes. Classification performance measures were calculated on an independent (unseen) test set. The area under the receiver operating characteristic curve (AUC), sensitivity (%), and specificity (%) were reported for each classifier. The logistic regression classifier demonstrated the highest AUC: 0.824 (sensitivity: 81.8%, specificity: 74.2%) for the TN sub-group and 0.778 (sensitivity: 71.4%, specificity: 71.6%) for the HER2 sub-group. ML classifiers demonstrate high diagnostic accuracy in classifying TN versus NTN and HER2+ versus HER2- breast cancers using US images. Identification of more aggressive breast cancer subtypes early in the diagnostic process could help achieve better prognoses by prioritizing clinical referral and prompting adequate early treatment.

Early MS Identification Using Non-linear Functional Connectivity and Graph-theoretic Measures of Cognitive Task-fMRI Data.

Functional neuroimaging has developed a fundamental ground for understanding the physical basis of the brain. Recent studies have extracted invaluable information from the underlying substrate of the brain. However, cognitive deficiency has insufficiently been assessed by researchers in multiple sclerosis (MS). Therefore, extracting the brain network differences among relapsing-remitting MS (RRMS) patients and healthy controls as biomarkers of cognitive task functional magnetic resonance imaging (fMRI) data and evaluating such biomarkers using machine learning were the aims of this study. In order to activate cognitive functions of the brain, blood-oxygen-level-dependent (BOLD) data were collected throughout the application of a cognitive task. Accordingly, a nonlinear-based brain network was established using kernel mutual information based on the automated anatomical labeling atlas (AAL). Subsequently, a statistical test was carried out to determine the variation in brain network measures between the two groups on binary adjacency matrices. We also found the prominent graph features by merging the Wilcoxon rank-sum test with the Fisher score as a hybrid feature selection method. The results of the classification performance measures showed that the construction of a brain network using a new nonlinear connectivity measure in task-fMRI performs better than the linear connectivity measures in terms of classification. The Wilcoxon rank-sum test also demonstrated a superior result for clinical applications. We believe that non-linear connectivity measures, like KMI, outperform linear connectivity measures, like correlation coefficient in finding the biomarkers of MS disease according to classification performance metrics. The performance of some brain regions (the hippocampus, parahippocampus, cuneus, pallidum, and two segments of the cerebellum) is different between healthy and MS people.Non-linear connectivity measures, such as Kernel mutual information, perform better than linear connectivity measures, such as correlation coefficient, in finding the biomarkers of MS disease. Multiple sclerosis (MS) can disrupt the function of the central nervous system. The function of brain network is impaired in these patients. In this study, we evaluated the change in brain network based on a non-linear connectivity measure using cognitive task-based fMRI data between MS patients and healthy controls. We used Kernel mutual information (KMI) and designed a graph network based on the results of connectivity analysis. The the paced auditory serial addition test was used to activate cognitive functions of the brain. The classification was employed for the results using different decision tree -based technique and support vector machine. KMI can be considered a valid measure of connectivity over linear measures, like the correlation coefficient. KMI does not have the drawbacks of mutual information technique. However, further studies should be implemented on brain data of MS patients to draw more definite conclusions.

The effect of feature extraction and data sampling on credit card fraud detection

Training a machine learning algorithm on a class-imbalanced dataset can be a difficult task, a process that could prove even more challenging under conditions of high dimensionality. Feature extraction and data sampling are among the most popular preprocessing techniques. Feature extraction is used to derive a richer set of reduced dataset features, while data sampling is used to mitigate class imbalance. In this paper, we investigate these two preprocessing techniques, using a credit card fraud dataset and four ensemble classifiers (Random Forest, CatBoost, LightGBM, and XGBoost). Within the context of feature extraction, the Principal Component Analysis (PCA) and Convolutional Autoencoder (CAE) methods are evaluated. With regard to data sampling, the Random Undersampling (RUS), Synthetic Minority Oversampling Technique (SMOTE), and SMOTE Tomek methods are evaluated. The F1 score and Area Under the Receiver Operating Characteristic Curve (AUC) metrics serve as measures of classification performance. Our results show that the implementation of the RUS method followed by the CAE method leads to the best performance for credit card fraud detection.

Intrusion detection approach for cloud and IoT environments using deep learning and Capuchin Search Algorithm

The Internet of Things (IoT) enabled technology will be adopted to develop smart cities, electronic commerce, electronic learning, electronic health, and other aspects of online activities. IoT enabled pervasive and wide connectivity to many objects and services. Therefore, it is easy to target IoT and cloud malware infection. Thus, cybersecurity is an essential problem to build robust IoT systems. This paper leverages the recent developments of the swarm intelligence (SI) algorithms combined with the advances of deep neural networks to build an efficient intrusion detection system for IoT-cloud based environments. First, deep neural networks are used to obtain optimal features from the IoT IDS data. Then, an efficient feature selection technique is proposed based on a recently developed SI optimizer called Capuchin Search Algorithm (CapSA). The performance of the developed model, called CNN-CapSA, is tested with four IoT-Cloud datasets, namely, NSL-KDD, BoT-IoT, KDD99, and CIC2017. Moreover, we consider extensive empirical comparisons to other optimization algorithms using several classification performance measures. The outcomes verified that the developed approach has a competitive performance overall datasets.

Determining Most Likely Links (MLL) for Network Fault Localization

We propose and evaluate a technique that learns the probability of a network transmission link experiencing a fault by using outlier flows (in the performance sense) as training data. This technique autonomously determines the most likely links causing performance degradation in a communications network; a critical feature of zero-touch network management. Our new Network Link Outlier Factor (NLOF) with most likely links (NLOF:MLL) is experimentally compared to the existing literature (including our original NLOF) using classification performance measures: recall, precision, F1-score, and time-to-detection. We utilize inferential statistics and a wide set of Mininet experiments to determine statistically significant performance differences. We find that our NLOF:MLL outperforms the existing literature wrt the important F1-score while exhibiting a competitive time-to-detection.

Fair evaluation of classifier predictive performance based on binary confusion matrix

Evaluating the ability of a classifier to make predictions on unseen data and increasing it by tweaking the learning algorithm are two of the main reasons motivating the evaluation of classifier predictive performance. In this study the behavior of Balanced AC1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$AC_1$$\\end{document} — a novel classifier accuracy measure — is investigated under different class imbalance conditions via a Monte Carlo simulation. The behavior of Balanced AC1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$AC_1$$\\end{document} is compared against that of several well-known performance measures based on binary confusion matrix. Study results reveal the suitability of Balanced AC1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$AC_1$$\\end{document} with both balanced and imbalanced data sets. A real example of the effects of class imbalance on the behavior of the investigated classifier performance measures is provided by comparing the performance of several machine learning algorithms in a churn prediction problem.