Classification Performance Metrics Research Articles

Integrating Convolutional Neural Network and Weighted Moving Average for Enhanced Human Fall Detection Performance

This study proposes an approach for human fall classification utilizing a combination of Weighted Moving Average (WMA) and Convolutional Neural Networks (CNN) on the SisFall dataset. Falls among elderly individuals pose a significant public health concern, necessitating effective automated detection systems for timely intervention and assistance. The SisFall dataset, comprising accelerometer data collected during simulated falls and activities of daily living, serves as the basis for training and evaluating the proposed classification system. The proposed method begins by preprocessing accelerometer data using a WMA technique to enhance signal quality and reduce noise. Subsequently, the preprocessed data are fed into a CNN architecture optimized for feature extraction and fall classification. The CNN leverages its ability to automatically learn discriminative features from raw sensor data, enabling robust and accurate classification of fall and non-fall events. Experimental results demonstrate the efficacy of the proposed approach in accurately distinguishing between fall and non-fall activities, achieving high classification performance metrics such as accuracy, precision, recall, and F1-score. Comparative analysis with existing methods showcases the WMA-CNN hybrid approach's superiority in classification accuracy and robustness. Overall, the proposed methodology presents a promising framework for real-time human fall classification using sensor data, offering potential applications in wearable devices, ambient assisted living systems, and healthcare monitoring technologies to enhance safety and well-being among elderly individuals.

A hybrid deep learning scheme for MRI-based preliminary multiclassification diagnosis of primary brain tumors.

The diagnosis and treatment of brain tumors have greatly benefited from extensive research in traditional radiomics, leading to improved efficiency for clinicians. With the rapid development of cutting-edge technologies, especially deep learning, further improvements in accuracy and automation are expected. In this study, we explored a hybrid deep learning scheme that integrates several advanced techniques to achieve reliable diagnosis of primary brain tumors with enhanced classification performance and interpretability. This study retrospectively included 230 patients with primary brain tumors, including 97 meningiomas, 66 gliomas and 67 pituitary tumors, from the First Affiliated Hospital of Yangtze University. The effectiveness of the proposed scheme was validated by the included data and a commonly used data. Based on super-resolution reconstruction and dynamic learning rate annealing strategies, we compared the classification results of several deep learning models. The multi-classification performance was further improved by combining feature transfer and machine learning. Classification performance metrics included accuracy (ACC), area under the curve (AUC), sensitivity (SEN), and specificity (SPE). In the deep learning tests conducted on two datasets, the DenseNet121 model achieved the highest classification performance, with five-test accuracies of 0.989 ± 0.006 and 0.967 ± 0.013, and AUCs of 0.999 ± 0.001 and 0.994 ± 0.005, respectively. In the hybrid deep learning tests, LightGBM, a promising classifier, achieved accuracies of 0.989 and 0.984, which were improved from the original deep learning scheme of 0.987 and 0.965. Sensitivities for both datasets were 0.985, specificities were 0.988 and 0.984, respectively, and relatively desirable receiver operating characteristic (ROC) curves were obtained. In addition, model visualization studies further verified the reliability and interpretability of the results. These results illustrated that deep learning models combining several advanced technologies can reliably improve the performance, automation, and interpretability of primary brain tumor diagnosis, which is crucial for further brain tumor diagnostic research and individualized treatment.

Prompt Design through ChatGPT’s Zero-Shot Learning Prompts: A Case of Cost-Sensitive Learning on a Water Potability Dataset

Datasets used in AI applications for human health require careful selection. In healthcare, machine learning (ML) models are fine-tuned to reduce errors, and our study focuses on minimizing errors by generating code snippets for cost-sensitive learning using water potability datasets. Water potability ensures safe drinking water through various scientific methods, with our approach using ML algorithms for prediction. We preprocess data with ChatGPT-generated code snippets and aim to demonstrate how zero-shot learning prompts in ChatGPT can produce reliable code snippets that cater to cost-sensitive learning. Our dataset is sourced from Kaggle. We compare model performance metrics of logistic regressors and gradient boosting classifiers without additional code fine-tuning to check the accuracy. Other classifier performance metrics are compared with results of the top 5 code authors on the Kaggle scoreboard. Cost-sensitive learning is crucial in domains like healthcare to prevent misclassifications with serious consequences, such as type II errors in water potability assessment.

Digital dual test syphilis/HIV detection based on Fourier Descriptors of Cyclic Voltammetry curves

Background:Effective and timely detection is vital for mitigating the severe impacts of Sexually Transmitted Infections (STI), including syphilis and HIV. Cyclic Voltammetry (CV) sensors have shown promise as diagnostic tools for these STI, offering a pathway towards cost-effective solutions in primary health care settings. Objective:This study aims to pioneer the use of Fourier Descriptors (FDs) in analyzing CV curves as 2D closed contours, targeting the simultaneous detection of syphilis and HIV. Methods:Raw CV signals are filtered, resampled, and transformed into 2D closed contours for FD extraction. Essential shape characteristics are captured through selected coefficients. A complementary geometrical analysis further extracts features like curve areas and principal axes lengths from CV curves. A Mahalanobis Distance Classifier is employed for differentiation between patient and control groups. Results:The evaluation of the proposed method revealed promising results with classification performance metrics such as Accuracy and F1-Score consistently achieving values rounded to 0.95 for syphilis and 0.90 for HIV. These results underscore the potential efficacy of the proposed approach in differentiating between patient and control samples for STI detection. Conclusion:By integrating principles from biosensors, signal processing, image processing, machine learning, and medical diagnostics, this study presents a comprehensive approach to enhance the detection of both syphilis and HIV. This setts the stage for advanced and accessible STI diagnostic solutions.

Early Warning System of Attrition in the BPO Industry Using Machine Learning Classification Models

Employee attrition is one of the factors affecting gross margin erosion in the BPO industry, the fastest-growing industry in the Philippines, due to hiring and training costs. The cost of employee attrition depends on the employee's role and salary/wage level. This study proposed shifting the retention approach from reactive to proactive with the use of an early warning system for employee attrition. The early warning system was powered by Machine Learning Classification Models. The data used in this study are employees hired in 2021 and 2022 from one of the Telco/Communication programs in the BPO Industry. The data attributes considered in this study are composed of Employee status, Employee Performance, Employee Satisfaction, Payroll, Time Off History, Schedule, and employee Observation data. The data is trained and tested in the classification models (Decision Trees, rule-based classification, naïve Bayes, KNN, Logistic Regression, and Random Forest). Models are evaluated using the classification performance metrics (AUC, Accuracy, Precision, Recall, and F1 Score). The model with the highest predictive accuracy is selected and deployed to produce employee classification (Risk of Termination, Neutral, and Positive). This study mainly helps the company reduce turnover and costs and increase gross margin with the help of the early warning system that can predict the status of the employees using significant indicators.

Comprehensive evaluation of classification: an empirical study on consequence prediction of construction accidents in China

Purpose Consequence prediction is an emerging topic in safety management concerning the severity outcome of accidents. In practical applications, it is usually implemented through supervised learning methods; however, the evaluation of classification results remains a challenge. The previous studies mostly adopted simplex evaluation based on empirical and quantitative assessment strategies. This paper aims to shed new light on the comprehensive evaluation and comparison of diverse classification methods through visualization, clustering and ranking techniques. Design/methodology/approach An empirical study is conducted using 9 state-of-the-art classification methods on a real-world data set of 653 construction accidents in China for predicting the consequence with respect to 39 carefully featured factors and accident type. The proposed comprehensive evaluation enriches the interpretation of classification results from different perspectives. Furthermore, the critical factors leading to severe construction accidents are identified by analyzing the coefficients of a logistic regression model. Findings This paper identifies the critical factors that significantly influence the consequence of construction accidents, which include accident type (particularly collapse), improper accident reporting and handling (E21), inadequate supervision engineers (O41), no special safety department (O11), delayed or low-quality drawings (T11), unqualified contractor (C21), schedule pressure (C11), multi-level subcontracting (C22), lacking safety examination (S22), improper operation of mechanical equipment (R11) and improper construction procedure arrangement (T21). The prediction models and findings of critical factors help make safety intervention measures in a targeted way and enhance the experience of safety professionals in the construction industry. Research limitations/implications The empirical study using some well-known classification methods for forecasting the consequences of construction accidents provides some evidence for the comprehensive evaluation of multiple classifiers. These techniques can be used jointly with other evaluation approaches for a comprehensive understanding of the classification algorithms. Despite the limitation of specific methods used in the study, the presented methodology can be configured with other classification methods and performance metrics and even applied to other decision-making problems such as clustering. Originality/value This study sheds new light on the comprehensive comparison and evaluation of classification results through visualization, clustering and ranking techniques using an empirical study of consequence prediction of construction accidents. The relevance of construction accident type is discussed with the severity of accidents. The critical factors influencing the accident consequence are identified for the sake of taking prevention measures for risk reduction. The proposed method can be applied to other decision-making tasks where the evaluation is involved as an important component.

Detecting Fake Online Reviews: An Unsupervised Detection Method With a Novel Performance Evaluation

ABSTRACT Fake reviews are critical issues in the online world, as they affect the credibility of e-commerce platforms and undermine consumers’ trust. Therefore, fake-review detection is of great significance. Since fake-review detection is an unsupervised problem, most existing methods and performance metrics cannot be applied. In addition, contemporary review manipulations are much more difficult to detect than before. To address these two problems, we first propose a fully unsupervised method with steps of survey research, fake-review feature analysis, fake index estimation, and fake-review selection. Fake-review features can be accurately derived from existing studies and survey research. Second, we propose a recommendation-based performance metric for evaluating fake-review detection methods. This metric differs from traditional binary classification performance metrics, as it can be used on review data with no objective review authenticity classifications. In this research, we utilize Dianping as a case study to evaluate the effectiveness of the proposed detection method and performance metric.

Assessment of Deep Learning Models for Cutaneous Leishmania Parasite Diagnosis Using Microscopic Images.

Cutaneous leishmaniasis (CL) is a common illness that causes skin lesions, principally ulcerations, on exposed regions of the body. Although neglected tropical diseases (NTDs) are typically found in tropical areas, they have recently become more common along Africa's northern coast, particularly in Libya. The devastation of healthcare infrastructure during the 2011 war and the following conflicts, as well as governmental apathy, may be causal factors associated with this catastrophic event. The main objective of this study is to evaluate alternative diagnostic strategies for recognizing amastigotes of cutaneous leishmaniasis parasites at various stages using Convolutional Neural Networks (CNNs). The research is additionally aimed at testing different classification models employing a dataset of ultra-thin skin smear images of Leishmania parasite-infected people with cutaneous leishmaniasis. The pre-trained deep learning models including EfficientNetB0, DenseNet201, ResNet101, MobileNetv2, and Xception are used for the cutaneous leishmania parasite diagnosis task. To assess the models' effectiveness, we employed a five-fold cross-validation approach to guarantee the consistency of the models' outputs when applied to different portions of the full dataset. Following a thorough assessment and contrast of the various models, DenseNet-201 proved to be the most suitable choice. It attained a mean accuracy of 0.9914 along with outstanding results for sensitivity, specificity, positive predictive value, negative predictive value, F1-score, Matthew's correlation coefficient, and Cohen's Kappa coefficient. The DenseNet-201 model surpassed the other models based on a comprehensive evaluation of these key classification performance metrics.

Deep learning-based burned forest areas mapping via Sentinel-2 imagery: a comparative study.

In order to evaluate the effects of forest fires on the dynamics of the function and structure of ecosystems, it is necessary to determine burned forest areas with high accuracy, effectively, economically, and practically using satellite images. Extraction of burned forest areas utilizing high-resolution satellite images and image classification algorithms and assessing the successfulness of varied classification algorithms has become a prominent research field. This study aims to indicate on the capability of the deep learning-based Stacked Autoencoders method for the burned forest areas mapping from Sentinel-2 satellite images. The Stacked Autoencoders, used in this study as an unsupervised learning method, were compared qualitatively and quantitatively with frequently used supervised learning algorithms (k-Nearest Neighbors (k-NN), Subspaced k-NN, Support Vector Machines, Random Forest, Bagged Decision Tree, Naive Bayes, Linear Discriminant Analysis) on two distinct burnt forest zones. By selecting burned forest zones with contrasting structural characteristics from one another, an objective assessment was achieved. Manually digitized burned areas from Sentinel-2 satellite images were utilized for accuracy assessment. For comparison, different classification performance and quality metrics (Overall Accuracy, Mean Squared Error, Correlation Coefficient, Structural Similarity Index Measure, Peak Signal-to-Noise Ratio, Universal Image Quality Index, and KAPPA metrics) were used. In addition, whether the Stacked Autoencoders method produces consistent results was examined through boxplots. In terms of both quantitative and qualitative analysis, the Stacked Autoencoders method showed the highest accuracy values.

Statistical Analysis of Imbalanced Classification with Training Size Variation and Subsampling on Datasets of Research Papers in Biomedical Literature

The overall purpose of this paper is to demonstrate how data preprocessing, training size variation, and subsampling can dynamically change the performance metrics of imbalanced text classification. The methodology encompasses using two different supervised learning classification approaches of feature engineering and data preprocessing with the use of five machine learning classifiers, five imbalanced sampling techniques, specified intervals of training and subsampling sizes, statistical analysis using R and tidyverse on a dataset of 1000 portable document format files divided into five labels from the World Health Organization Coronavirus Research Downloadable Articles of COVID-19 papers and PubMed Central databases of non-COVID-19 papers for binary classification that affects the performance metrics of precision, recall, receiver operating characteristic area under the curve, and accuracy. One approach that involves labeling rows of sentences based on regular expressions significantly improved the performance of imbalanced sampling techniques verified by performing statistical analysis using a t-test documenting performance metrics of iterations versus another approach that automatically labels the sentences based on how the documents are organized into positive and negative classes. The study demonstrates the effectiveness of ML classifiers and sampling techniques in text classification datasets, with different performance levels and class imbalance issues observed in manual and automatic methods of data processing.

Coarsening effects on k-partite network classification

The growing data size poses challenges for storage and computational processing time in semi-supervised models, making their practical application difficult; researchers have explored the use of reduced network versions as a potential solution. Real-world networks contain diverse types of vertices and edges, leading to using k-partite network representation. However, the existing methods primarily reduce uni-partite networks with a single type of vertex and edge. We develop a new coarsening method applicable to the k-partite networks that maintain classification performance. The empirical analysis of hundreds of thousands of synthetically generated networks demonstrates the promise of coarsening techniques in solving large networks’ storage and processing problems. The findings indicate that the proposed coarsening algorithm achieved significant improvements in storage efficiency and classification runtime, even with modest reductions in the number of vertices, leading to over one-third savings in storage and twice faster classifications; furthermore, the classification performance metrics exhibited low variation on average.

Machine Learning-Based Decision Tree for Identifying and Grading Severity of Immune Effector Cell-Associated Neurotoxicity Syndrome (ICANS) Based on Neurological Adverse Events in Patients Treated with CD19 CAR-T

Introduction One of the major challenges associated with Chimeric Antigen Receptor T-cell (CAR-T) therapy is the development of immune effector cell-associated neurotoxicity syndrome (ICANS), which can vary in severity and requires prompt intervention. This study proposes using machine learning in combination with clinical insight to build a decision tree that accurately identifies and grades ICANS severity in patients treated with CD19 CAR-T based on the neurological adverse events (NAEs). Methods We conducted a retrospective analysis of 384 patients that exhibited NAEs after being treated with CD19-targeted CART-cell therapy in the pooled clinical trial data from the Medidata Enterprise Data Store. Of note, this dataset excludes patients without NAEs; these patients were assumed to not experience clinical ICANS. Patient NAEs were reviewed and the highest ICANS grade was assigned by a neuroimmunologist, resulting in the following patient counts: grade 0 (n=61), 1 (n=69), 2 (n=166), 3 (n=75), 4 (n=13). The clinical expert graded each patient based on the NAEs recorded in the first 28 days post infusion during the clinical trial, their CTCAE toxicity and symptom onset day. To process NAEs, we first mapped each of the 219 verbatim terms to the MedDRA hierarchy using a proprietary Medidata coding algorithm. We then aggregated the NAEs to the High Level Group Term (HLGT) level except for the MedDRA term “Neurological Disorders NEC” which was broken down to the High Level Term (HLT) level due to the large size of this group (73 out of 219 terms).The mapping of NAEs to MedDRA groups was reviewed together with the neuroimmunologist. The final dataset included the CTCAE severity of the NAEs mapped to MedDRA groups as features and the expert-labeled ICANS grades as the target variable. The data was then split into train (70%) and test (30%) sets stratifying by study and ICANS grade. We fit a decision tree while performing 5-fold hyperparameter grid search on the train set. The tree with the highest area under the Receiver Operating Characteristic curve one-vs-one (ROCAUC OVO) was selected and evaluated on the test set using an array of classification performance metrics. The model output was evaluated at the ICANS grade and severity (no/moderate/severe) level. The confusion matrices for the test set and the full decision tree were plotted. Results When binning the ICANS grade prediction by severity (grade 0 = no ICANS, grade 1-2 = moderate, grade 3-4 = severe), the model achieves a strong out-of-sample accuracy of 83.6% and ROCAUC OVO of 0.935. Regarding the individual ICANS grade prediction, the model yielded lower out-of-sample accuracy of 54.3%, with the majority of errors between grades 1 and 2. In spite of this, there was still reasonable discrimination based on the out-of-sample ROCAUC OVO values of 0.863. The training algorithm selected six MedDRA groups of NAEs to make predictions with: Headaches, Seizures (incl subtypes), Encephalopathies, Cortical dysfunction NEC, Disturbances in consciousness NEC, and Movement disorders (incl parkinsonism). We observe that patients with only headaches (a separate feature in our dataset) are all labeled as having no ICANS while those with seizures of grade 2 and higher are classified as grade 4. Refer to Figure 1 for the full decision tree. Comparing the neurologist-assigned versus predicted ICANS grades in the out-of-sample patient group (Figure 2) revealed that the majority of errors were bound to +/- one grade. Only three out of 116 patients were mislabeled by two grades. Crucially, none of the grade 1+ patients were predicted by the model to have grade 0. Conclusion Our machine learning-based decision tree accurately predicts the severity and grade of ICANS in CD19 CAR-T treated patients, aiding in clinical decision-making as well as retrospective analyses of ICANS patients. This type of clinical decision support tool has the potential to be a powerful aid to physicians by providing them with earlier physiological insights and data-driven assessments to help them triage patients accordingly. The model's performance highlights its potential clinical utility in improving patient care and outcomes. Further validation across multiple neurologists and prospective studies are warranted to assess its generalizability and refine its predictive capabilities. Acknowledgments We thank the developers of Scikit-learn and Dtreeviz Python packages for open-sourcing their code.

A membership-based resampling and cleaning algorithm for multi-class imbalanced overlapping data

Real-world datasets frequently have an imbalanced class distribution, which significantly degrades classification performance. However, some studies have suggested that the adverse effects of class imbalance occur only when datasets have other intrinsic characteristics (such as class overlap, noise, and data scarcity). Noise and class overlap have the greatest effect. To deal with other intrinsic characteristics that affect classification performance in a multi-class environment, such as class overlap, noise, and data scarcity, we propose a method that can directly handle multi-class overlapping data, called the membership-based multi-class resampling and cleaning (MC-MBRC) algorithm. The proposed method divides samples into safe, overlapping, and noisy areas based on their membership degrees, and then according to the influence of the samples in each area on classification performance, it performs various operations such as noise removal, interpolation oversampling, and energy-based cleaning of the overlapping region. An extensive comparison using various datasets shows that, compared with state-of-the-art methods, the proposed method makes significant statistical improvements in different classification performance metrics and is robust to data containing class overlap and label noise. Furthermore, for datasets that do not have complex intrinsic features, MC -MBRC will not significantly degrade classification performance.

Coreference Resolution for Improving Performance Measures of Classification Tasks

There are several possibilities to improve classification in natural language processing tasks. In this article, we focused on the issue of coreference resolution that was applied to a manually annotated dataset of true and fake news. This dataset was used for the classification task of fake news detection. The research aimed to determine whether performing coreference resolution on the input data before classification or classifying them without performing coreference resolution is more effective. We also wanted to verify whether it is possible to enhance classifier performance metrics by incorporating coreference resolution into the data preparation process. A methodology was proposed, in which we described the implementation methods in detail, starting from the identification of entity mentions in the text using the neuralcoref algorithm, then through word-embedding models (TF–IDF, Doc2Vec), and finally to several machine learning methods. The result was a comparison of the implemented classifiers based on the performance metrics described in the theoretical part. The best result for accuracy was observed for the dataset with coreference resolution applied, which had a median value of 0.8149, while for the F1 score, the best result had a median value of 0.8101. However, the more important finding is that the processed data with the application of coreference resolution led to an improvement in performance metrics in the classification tasks.

Detection of Simultaneous Bearing Faults Fusing Cross Correlation With Multikernel SVM

Detection of simultaneous bearing faults for condition monitoring (CM) of bearings using time-domain analysis is quite challenging and open area particularly in noisy environment. This work presents a new scheme for simultaneous bearing fault detection using vibration signal, in cases where single-point localized bearing fault and multiple-point compound fault coexist. Bearings of a 415 V, 3kW, 3-phase Squirrel Cage Induction Motor (SCIM) have been used for data collection, while the loading arrangement is done using a 110V, 4kW DC generator connected with a load box and coupled to the motor. A cross-correlation-based time-domain feature extraction approach has been introduced. The neighborhood component analysis (NCA) technique has been applied to the cross-correlation-based features to reduce the complexity of the proposed model. Furthermore, the selected features have been fed into a multi-kernel support vector machine (MKSVM) to classify simultaneous bearing faults. This method has also been tested on signals contaminated with white Gaussian noise to verify reliability in the industrial environment. It is found that with only five features, the proposed model yields 100% classification performance metrics for raw signal and under noisy environments with signal-to-noise ratio (SNR) of 20 dB to 50 dB for both full load and no-load conditions. Whereas at 10 dB SNR value, performance decreases slightly, still an overall classification performance metrics of more than 99% is achieved by this method. Furthermore, this method has enhanced performance when compared to earlier studies with publicly available databases for localized bearing failure identification.

Batch-balanced focal loss: a hybrid solution to class imbalance in deep learning.

To validate the effectiveness of an approach called batch-balanced focal loss (BBFL) in enhancing convolutional neural network (CNN) classification performance on imbalanced datasets. BBFL combines two strategies to tackle class imbalance: (1)batch-balancing to equalize model learning of class samples and (2) focal loss to add hard-sample importance to the learning gradient. BBFL was validated on two imbalanced fundus image datasets: a binary retinal nerve fiber layer defect (RNFLD) dataset () and a multiclass glaucoma dataset (). BBFL was compared to several imbalanced learning techniques, including random oversampling (ROS), cost-sensitive learning, and thresholding, based on three state-of-the-art CNNs. Accuracy, F1-score, and the area under the receiver operator characteristic curve (AUC) were used as the performance metrics for binary classification. Mean accuracy and mean F1-score were used for multiclass classification. Confusion matrices, t-distributed neighbor embedding plots, and GradCAM were used for the visual assessment of performance. In binary classification of RNFLD, BBFL with InceptionV3 (93.0% accuracy, 84.7% F1, 0.971 AUC) outperformed ROS (92.6% accuracy, 83.7% F1, 0.964 AUC), cost-sensitive learning (92.5% accuracy, 83.8% F1, 0.962 AUC), and thresholding (91.9% accuracy, 83.0% F1, 0.962 AUC) and others. In multiclass classification of glaucoma, BBFL with MobileNetV2 (79.7% accuracy, 69.6% average F1 score) outperformed ROS (76.8% accuracy, 64.7% F1), cost-sensitive learning (78.3% accuracy, 67.8.8% F1), and random undersampling (76.5% accuracy, 66.5% F1). The BBFL-based learning method can improve the performance of a CNN model in both binary and multiclass disease classification when the data are imbalanced.

Deep-discovery: Anomaly discovery in software-defined networks using artificial neural networks

This paper proposes Deep-Discovery, an Intrusion Detection System (IDS), to perform Anomaly Discovery in Software-Defined Networking (SDN) using Artificial Neural Network (ANN). The proposed IDS framework utilizes the Multi-Layer Perceptron (MLP), a Feedforward (FF) ANN, to detect volume-based and protocol-based Distributed Denial of Service (DDoS) attacks on the data plane of SDN. The proposed model considers the attack detection a multi-class classification problem and classifies the network traffic into six attack classes with an accuracy of 98.81% and a minimal False Alarm Rate (FAR) of 0.002. The proposed classification model addresses the binary classification problem to compare and analyze the classification performance metrics. The Deep-Discovery that deals with the binary classification problem categorizes the network traffic into anomalous and normal traffic with 99.79% accuracy and a nominal FAR of 0.0001. The novelty of this work is its emphasis on obtaining the optimal performance metrics with a simple neural network with minimal computational overhead rather than an intricate and complex model.

Artificial intelligence algorithm developed to predict immune checkpoint inhibitors efficacy in non–small-cell lung cancer.

9132 Background: Many non-small cell lung cancer (NSCLC) patients with high PD-L1 IHC expression and/or high TMB level do not respond to immune-check point inhibitors (ICIs). Discovery of novel predictive biomarkers for ICI response in NSCLC continues to be a critical unmet medical need, given the expense, potential toxicity, and therapeutic delay due to ICI treatment without benefit. Methods: We developed a supervised deep learning algorithm (Deep-IO) to predict therapeutic response to ICIs in NSCLC patients from standard histology whole slide hematoxylin and eosin (H&E) images, which was trained based on ICI objective response rate. We utilized a convolutional neural network (CNN) implemented in Pytorch. The efficiency of the Deep-IO was evaluated through classification performance metrics and the area under the receiver operating characteristic curve (AUROC). The algorithm was trained and tested on 85218 tiles (size: 512x512 pixels) in 446 advanced stage NSCLC patients from the Dana Farber Cancer Institute (DFCI), all treated with ICI monotherapy. Results: The objective response rate, comprising complete and partial response, was 26% (n = 114) in the overall cohort. The DFCI patient cohort was randomly split into a training set (N=379, 85%) and a test set (N=67, 15%). The classifier’s predicted class (responder vs non-responder) probabilities at the tiles level were averaged for each patient. Quantitative evaluation of whole section test dataset shows that the developed model achieves the accuracy of 0.72, weighted average (WAVG) precision of 0.77, WAVG recall of 0.72 and WAVG F-score of 0.74 on classifying ICI non-responders from responders. Comparing with PD-L1 expression (TPS%) and TMB (mu/Mb), Deep-IO had superior predictive power for ICI response (AUROC=0.75) in the test set. The combined Deep-IO+PD-L1 scores resulted in a significantly greater AUROC value of 0.82 compared to single tests and a 32% improvement in specificity (0.88 vs. 0.56) compared to PD-L1 evaluation alone (Table). Conclusions: This proof-of-concept study shows that an artificial intelligence–powered H&E image classifier can predict ICI effectiveness in NSCLC. Deep-IO outperformed both established predictive biomarkers. We will assess Deep-IO in additional external NSCLC data sets for validation and confirmation. [Table: see text]

Deep Learning Using Preoperative AS-OCT Predicts Graft Detachment in DMEK.

To evaluate a novel deep learning algorithm to distinguish between eyes that may or may not have a graft detachment based on pre-Descemet membrane endothelial keratoplasty (DMEK) anterior segment optical coherence tomography (AS-OCT) images. Retrospective cohort study. A multiple-instance learning artificial intelligence (MIL-AI) model using a ResNet-101 backbone was designed. AS-OCT images were split into training and testing sets. The MIL-AI model was trained and validated on the training set. Model performance and heatmaps were calculated from the testing set. Classification performance metrics included F1 score (harmonic mean of recall and precision), specificity, sensitivity, and area under curve (AUC). Finally, MIL-AI performance was compared to manual classification by an experienced ophthalmologist. In total, 9466 images of 74 eyes (128 images per eye) were included in the study. Images from 50 eyes were used to train and validate the MIL-AI system, while the remaining 24 eyes were used as the test set to determine its performance and generate heatmaps for visualization. The performance metrics on the test set (95% confidence interval) were as follows: F1 score, 0.77 (0.57-0.91); precision, 0.67 (0.44-0.88); specificity, 0.45 (0.15-0.75); sensitivity, 0.92 (0.73-1.00); and AUC, 0.63 (0.52-0.86). MIL-AI performance was more sensitive (92% vs. 31%) but less specific (45% vs. 64%) than the ophthalmologist's performance. The MIL-AI predicts with high sensitivity the eyes that may have post-DMEK graft detachment requiring rebubbling. Larger-scale clinical trials are warranted to validate the model. MIL-AI models represent an opportunity for implementation in routine DMEK suitability screening.

Artificial Intelligence Technologies in Cardiology

As the world produces exabytes of data, there is a growing need to find new methods that are more suitable for dealing with complex datasets. Artificial intelligence (AI) has significant potential to impact the healthcare industry, which is already on the road to change with the digital transformation of vast quantities of information. The implementation of AI has already achieved success in the domains of molecular chemistry and drug discoveries. The reduction in costs and in the time needed for experiments to predict the pharmacological activities of new molecules is a milestone in science. These successful applications of AI algorithms provide hope for a revolution in healthcare systems. A significant part of artificial intelligence is machine learning (ML), of which there are three main types-supervised learning, unsupervised learning, and reinforcement learning. In this review, the full scope of the AI workflow is presented, with explanations of the most-often-used ML algorithms and descriptions of performance metrics for both regression and classification. A brief introduction to explainable artificial intelligence (XAI) is provided, with examples of technologies that have developed for XAI. We review important AI implementations in cardiology for supervised, unsupervised, and reinforcement learning and natural language processing, emphasizing the used algorithm. Finally, we discuss the need to establish legal, ethical, and methodical requirements for the deployment of AI models in medicine.