Precision-recall Curve Research Articles

Development and validation of a machine learning model for prediction of comorbid major depression disorder among narcolepsy type 1

BackgroundMajor depression disorder (MDD) forms a common psychiatric comorbidity among patients with narcolepsy type 1 (NT1), yet its impact on patients with NT1 is often overlooked by neurologists. Currently, there is a lack of effective methods for accurately predicting MDD in patients with NT1. ObjectiveThis study utilized machine learning (ML) algorithms to identify critical variables and developed the prediction model for predicting MDD in patients with NT1. MethodsThe study included 267 NT1 patients from four sleep centers. The diagnosis of comorbid MDD was based on Diagnostic and Statistical Manual of Mental Disorders fifth edition (DSM-5). ML models, including six full models and six compact models, were developed using a training set. The performance of these models was compared in the testing set, and the optimal model was evaluated in the testing set. Various evaluation metrics, such as Area under the receiver operating curve (AUC), precision-recall (PR) curve and calibration curve were employed to assess and compare the performance of the ML models. Model interpretability was demonstrated using SHAP. ResultIn the testing set, the logistic regression (LG) model demonstrated superior performance compared to other ML models based on evaluation metrics such as AUC, PR curve, and calibration curve. The top eight features used in the LG model, ranked by feature importance, included social impact scale (SIS) score, narcolepsy severity scale (NSS) score, total sleep time, body mass index (BMI), education years, age of onset, sleep efficiency, sleep latency. ConclusionThe study yielded a straightforward and practical ML model for the early identification of MDD in patients with NT1. A web-based tool for clinical applications was developed, which deserves further verification in diverse clinical settings.

Improving Clinician Performance in Classifying EEG Patterns on the Ictal-Interictal Injury Continuum Using Interpretable Machine Learning.

In intensive care units (ICUs), critically ill patients are monitored with electroencephalography (EEG) to prevent serious brain injury. EEG monitoring is constrained by clinician availability, and EEG interpretation can be subjective and prone to interobserver variability. Automated deep-learning systems for EEG could reduce human bias and accelerate the diagnostic process. However, existing uninterpretable (black-box) deep-learning models are untrustworthy, difficult to troubleshoot, and lack accountability in real-world applications, leading to a lack of both trust and adoption by clinicians. We developed an interpretable deep-learning system that accurately classifies six patterns of potentially harmful EEG activity - seizure, lateralized periodic discharges (LPDs), generalized periodic discharges (GPDs), lateralized rhythmic delta activity (LRDA), generalized rhythmic delta activity (GRDA), and other patterns - while providing faithful case-based explanations of its predictions. The model was trained on 50,697 total 50-second continuous EEG samples collected from 2711 patients in the ICU between July 2006 and March 2020 at Massachusetts General Hospital. EEG samples were labeled as one of the six EEG patterns by 124 domain experts and trained annotators. To evaluate the model, we asked eight medical professionals with relevant backgrounds to classify 100 EEG samples into the six pattern categories - once with and once without artificial intelligence (AI) assistance - and we assessed the assistive power of this interpretable system by comparing the diagnostic accuracy of the two methods. The model's discriminatory performance was evaluated with area under the receiver-operating characteristic curve (AUROC) and area under the precision-recall curve. The model's interpretability was measured with task-specific neighborhood agreement statistics that interrogated the similarities of samples and features. In a separate analysis, the latent space of the neural network was visualized by using dimension reduction techniques to examine whether the ictal-interictal injury continuum hypothesis, which asserts that seizures and seizure-like patterns of brain activity lie along a spectrum, is supported by data. The performance of all users significantly improved when provided with AI assistance. Mean user diagnostic accuracy improved from 47 to 71% (P<0.04). The model achieved AUROCs of 0.87, 0.93, 0.96, 0.92, 0.93, and 0.80 for the classes seizure, LPD, GPD, LRDA, GRDA, and other patterns, respectively. This performance was significantly higher than that of a corresponding uninterpretable black-box model (with P<0.0001). Videos traversing the ictal-interictal injury manifold from dimension reduction (a two-dimensional representation of the original high-dimensional feature space) give insight into the layout of EEG patterns within the network's latent space and illuminate relationships between EEG patterns that were previously hypothesized but had not yet been shown explicitly. These results indicate that the ictal-interictal injury continuum hypothesis is supported by data. Users showed significant pattern classification accuracy improvement with the assistance of this interpretable deep-learning model. The interpretable design facilitates effective human-AI collaboration; this system may improve diagnosis and patient care in clinical settings. The model may also provide a better understanding of how EEG patterns relate to each other along the ictal-interictal injury continuum. (Funded by the National Science Foundation, National Institutes of Health, and others.).

Impact Exploration of Spatiotemporal Feature Derivation and Selection on Machine Learning-Based Predictive Models for Post-Embolization Cerebral Aneurysm Recanalization.

To enhance the performance of machine learning (ML) models for the post-embolization recanalization of cerebral aneurysms, we evaluated the impact of hemodynamic feature derivation and selection method on six ML algorithms. We utilized computational fluid dynamics (CFD) to simulate hemodynamics in 66 cerebral aneurysms from 65 patients, including 57 stable and nine recanalized aneurysms. We derived a total of 107 features for each aneurysm, encompassing four clinical features, 12 morphological features, and 91 hemodynamic features. To investigate the influence of feature derivation and selection methods on the ML models, we employed two derivation methods, simplified and fully derived, in combination with four selection methods: all features, statistically significant analysis, stepwise multivariate logistic regression analysis (stepwise-LR), and recursive feature elimination (RFE). Model performance was assessed using the area under the receiver operating characteristic curve (AUROC) and precision-recall curve (AUPRC) on both the training and testing datasets. The AUROC values on the testing dataset exhibited a wide-ranging spectrum, spanning from 0.373 to 0.863. Fully derived features and the RFE selection method demonstrated superior performance in intra-model comparisons. The multi-layer perceptron (MLP) model, trained with RFE-selected fully derived features, achieved the best performance on the testing dataset, with an AUROC value of 0.863 (95% CI: 0.684- 1.000). Our study demonstrated the importance of feature derivation and selection in determining the performance of ML models. This enabled the development of accurate decision-making models without the need to invade the patient.

Ensemble Learning-based Brain Stroke Prediction Model Using Magnetic Resonance Imaging

Brain stroke (BS) imposes a substantial burden on healthcare systems due to the long-term care and high expenditure. Earlier detection and intervention can reduce the impact of BS. Magnetic resonance imaging (MRI) is commonly applied for BS detection. Deep learning techniques can employ MRI images to identify the BS risks in the initial stages. This study developed a BS detection model using an ensemble learning approach that combines the predictions of the base models. A MobileNet V3 model backbone was used to extract the intricate patterns of BS from MRI images. LightGBM and CatBoost models were used as base models to predict BS using the extracted features. In addition, the random forest model was used to integrate the predictions of base models to identify BS. The proposed model was generalized on a public MRI dataset that covers 2888 clinical MRI images. The experimental outcomes showed the effectiveness of the suggested BS detection model. The proposed model has obtained an accuracy of 98.7%, an area under the receiver operating characteristic score of 0.95, and an area under the precision–recall curve of 0.92. The recommended model is believed to be deployed in real-time healthcare settings to assist radiologists and clinicians in making effective decisions.

Predicting 1year readmission for heart failure: A comparative study of machine learning and the LACE index.

There is a lack of tools for accurately identifying the risk of readmission for heart failure in elderly patients with arrhythmia. The aim of this study was to establish and compare the performance of the LACE [length of stay ('L'), acute (emergent) admission ('A'), Charlson comorbidity index ('C') and visits to the emergency department during the previous 6months ('E')] index and machine learning in predicting 1 year readmission for heart failure in elderly patients with arrhythmia. Elderly patients with arrhythmia who were hospitalized at Sichuan Provincial People's Hospital between 1 June 2018 and 31 May 2020 were enrolled. The LACE index was calculated for each patient, and the area under the receiver operating characteristic curve (AUROC) was calculated. Six machine learning algorithms, combined with three variable selection methods and clinically relevant features available at the time of hospital discharge, were used to develop machine learning models. AUROC and area under the precision-recall curve (AUPRC) were used to assess discrimination. Shapley additive explanations (SHAP) analysis was used to explain the contributions of the features. A total of 523 patients were enrolled, and 108 patients experienced 1year hospital readmission for heart failure. The AUROC of the LACE index was 0.5886. The complete machine learning model had the best predictive performance, with an AUROC of 0.7571 and an AUPRC of 0.4096. The most important predictors for 1year readmission were educational level, total triiodothyronine (TT3), aspartate aminotransferase/alanine aminotransferase (AST/ALT), number of medications (NOM) and triglyceride (TG) level. Compared with the LACE index, the machine learning model can accurately identify the risk of 1year readmission for heart failure in elderly patients with arrhythmia.

Machine learning models on a web application to predict short-term postoperative outcomes following anterior cervical discectomy and fusion

BackgroundThe frequency of anterior cervical discectomy and fusion (ACDF) has increased up to 400% since 2011, underscoring the need to preoperatively anticipate adverse postoperative outcomes given the procedure’s expanding use. Our study aims to accomplish two goals: firstly, to develop a suite of explainable machine learning (ML) models capable of predicting adverse postoperative outcomes following ACDF surgery, and secondly, to embed these models in a user-friendly web application, demonstrating their potential utility.MethodsWe utilized data from the National Surgical Quality Improvement Program database to identify patients who underwent ACDF surgery. The outcomes of interest were four short-term postoperative adverse outcomes: prolonged length of stay (LOS), non-home discharges, 30-day readmissions, and major complications. We utilized five ML algorithms - TabPFN, TabNET, XGBoost, LightGBM, and Random Forest - coupled with the Optuna optimization library for hyperparameter tuning. To bolster the interpretability of our models, we employed SHapley Additive exPlanations (SHAP) for evaluating predictor variables’ relative importance and used partial dependence plots to illustrate the impact of individual variables on the predictions generated by our top-performing models. We visualized model performance using receiver operating characteristic (ROC) curves and precision-recall curves (PRC). Quantitative metrics calculated were the area under the ROC curve (AUROC), balanced accuracy, weighted area under the PRC (AUPRC), weighted precision, and weighted recall. Models with the highest AUROC values were selected for inclusion in a web application.ResultsThe analysis included 57,760 patients for prolonged LOS [11.1% with prolonged LOS], 57,780 for non-home discharges [3.3% non-home discharges], 57,790 for 30-day readmissions [2.9% readmitted], and 57,800 for major complications [1.4% with major complications]. The top-performing models, which were the ones built with the Random Forest algorithm, yielded mean AUROCs of 0.776, 0.846, 0.775, and 0.747 for predicting prolonged LOS, non-home discharges, readmissions, and complications, respectively.ConclusionsOur study employs advanced ML methodologies to enhance the prediction of adverse postoperative outcomes following ACDF. We designed an accessible web application to integrate these models into clinical practice. Our findings affirm that ML tools serve as vital supplements in risk stratification, facilitating the prediction of diverse outcomes and enhancing patient counseling for ACDF.

BiTGNN: Prediction of drug–target interactions based on bidirectional transformer and graph neural network on heterogeneous graph

Drug–target interaction (DTI) is a widely explored topic in the field of bioinformatics and plays a pivotal role in drug discovery. However, the traditional bio-experimental process of drug–target interaction identification requires a large investment of time and labor. To address this challenge, graph neural network (GNN) approaches in deep learning are becoming a prominent trend in the field of DTI research, which is characterized by multimodal processing of data, feature learning and interpretability in DTI. Nevertheless, some methods are still limited by homogeneous graphs and single features. To address the problems, we mechanistically analyze graph convolutional neural networks (GCNs) and graph attentional neural networks (GATs) to propose a new model for the prediction of drug–target interactions using graph neural networks named BiTGNN [Bidirectional Transformer (Bi-Transformer)–graph neural network]. The method first establishes drug–target pairs through the pseudo-position specificity scoring matrix (PsePSSM) and drug fingerprint data, and constructs a heterogeneous network by utilizing the relationship between the drug and the target. Then, the computational extraction of drug and target attributes is performed using GCNs and GATs for the purpose of model information flow extension and graph information enhancement. We collect interaction data using the proposed Bi-Transformer architecture, in which we design a bidirectional cross-attention mechanism for calculating the effects of drug–target interactions for realistic biological interaction simulations. Finally, a feed-forward neural network is used to obtain the feature matrices of the drug and the target, and DTI prediction is performed by fusing the two feature matrices. The Enzyme, Ion Channel (IC), G Protein-coupled Receptor (GPCR) and Nuclear Receptor (NR) datasets are used in the experiments, and compared with several existing mainstream models, our model outperforms in Area Under the ROC Curve (AUC), Specificity, Accuracy and the metric Area Under the Precision–Recall Curve (AUPR).

Evaluating the accuracy of a state-of-the-art large language model for prediction of admissions from the emergency room.

Artificial intelligence (AI) and large language models (LLMs) can play a critical role in emergency room operations by augmenting decision-making about patient admission. However, there are no studies for LLMs using real-world data and scenarios, in comparison to and being informed by traditional supervised machine learning (ML) models. We evaluated the performance of GPT-4 for predicting patient admissions from emergency department (ED) visits. We compared performance to traditional ML models both naively and when informed by few-shot examples and/or numerical probabilities. We conducted a retrospective study using electronic health records across 7 NYC hospitals. We trained Bio-Clinical-BERT and XGBoost (XGB) models on unstructured and structured data, respectively, and created an ensemble model reflecting ML performance. We then assessed GPT-4 capabilities in many scenarios: through Zero-shot, Few-shot with and without retrieval-augmented generation (RAG), and with and without ML numerical probabilities. The Ensemble ML model achieved an area under the receiver operating characteristic curve (AUC) of 0.88, an area under the precision-recall curve (AUPRC) of 0.72 and an accuracy of 82.9%. The naïve GPT-4's performance (0.79 AUC, 0.48 AUPRC, and 77.5% accuracy) showed substantial improvement when given limited, relevant data to learn from (ie, RAG) and underlying ML probabilities (0.87 AUC, 0.71 AUPRC, and 83.1% accuracy). Interestingly, RAG alone boosted performance to near peak levels (0.82 AUC, 0.56 AUPRC, and 81.3% accuracy). The naïve LLM had limited performance but showed significant improvement in predicting ED admissions when supplemented with real-world examples to learn from, particularly through RAG, and/or numerical probabilities from traditional ML models. Its peak performance, although slightly lower than the pure ML model, is noteworthy given its potential for providing reasoning behind predictions. Further refinement of LLMs with real-world data is necessary for successful integration as decision-support tools in care settings.

Towards proactive palliative care in oncology: developing an explainable EHR-based machine learning model for mortality risk prediction

BackgroundEx-ante identification of the last year in life facilitates a proactive palliative approach. Machine learning models trained on electronic health records (EHR) demonstrate promising performance in cancer prognostication. However, gaps in literature include incomplete reporting of model performance, inadequate alignment of model formulation with implementation use-case, and insufficient explainability hindering trust and adoption in clinical settings. Hence, we aim to develop an explainable machine learning EHR-based model that prompts palliative care processes by predicting for 365-day mortality risk among patients with advanced cancer within an outpatient setting.MethodsOur cohort consisted of 5,926 adults diagnosed with Stage 3 or 4 solid organ cancer between July 1, 2017, and June 30, 2020 and receiving ambulatory cancer care within a tertiary center. The classification problem was modelled using Extreme Gradient Boosting (XGBoost) and aligned to our envisioned use-case: “Given a prediction point that corresponds to an outpatient cancer encounter, predict for mortality within 365-days from prediction point, using EHR data up to 365-days prior.” The model was trained with 75% of the dataset (n = 39,416 outpatient encounters) and validated on a 25% hold-out dataset (n = 13,122 outpatient encounters). To explain model outputs, we used Shapley Additive Explanations (SHAP) values. Clinical characteristics, laboratory tests and treatment data were used to train the model. Performance was evaluated using area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC), while model calibration was assessed using the Brier score.ResultsIn total, 17,149 of the 52,538 prediction points (32.6%) had a mortality event within the 365-day prediction window. The model demonstrated an AUROC of 0.861 (95% CI 0.856–0.867) and AUPRC of 0.771. The Brier score was 0.147, indicating slight overestimations of mortality risk. Explanatory diagrams utilizing SHAP values allowed visualization of feature impacts on predictions at both the global and individual levels.ConclusionOur machine learning model demonstrated good discrimination and precision-recall in predicting 365-day mortality risk among individuals with advanced cancer. It has the potential to provide personalized mortality predictions and facilitate earlier integration of palliative care.

An intelligent model for classifying supraventricular tachycardia mechanisms based on 12-lead wearable electrocardiogram devices

To develop an intelligent model for differential diagnosis of atrioventricular nodal re-entrant tachycardia (AVNRT) and atrioventricular re-entrant tachycardia (AVRT) using 12-lead wearable electrocardiogram devices. A total of 356 samples of 12-lead supraventricular tachycardia (SVT) electrocardiograms recorded by wearable devices were randomly divided into training and validation sets using 5-fold cross validation to establish the intelligent classification model, and 101 patients with the diagnosis of SVT undergoing electrophysiological studies and radiofrequency ablation from October, 2021 to March, 2023 were selected as the testing set. The changes in electrocardiogram parameters before and during induced tachycardia were compared. Based on multiscale deep neural network, an intelligent diagnosis model for classifying SVT mechanisms was constructed and validated. The 3-lead electrocardiogram signals from Ⅱ, Ⅲ, and Ⅴ1 were extracted to build new classification models, whose diagnostic efficacy was compared with that of the 12-lead model. Of the 101 patients with SVT in the testing set, 68 were diagnosed with AVNRT and 33 were diagnosed with AVRT by electrophysiological study. The pre-trained model achieved a high area under the precision-recall curve (0.9492) and F1 score (0.8195) for identifying AVNRT in the validation set. The total F1 scores of the lead Ⅱ, Ⅲ, Ⅴ1, 3-lead and 12-lead intelligent diagnostic models in the testing set were 0.5597, 0.6061, 0.3419, 0.6003 and 0.6136, respectively. Compared with the 12-lead classification model, the lead-Ⅲ model had a net reclassification index improvement of -0.029 (P=0.878) and an integrated discrimination index improvement of -0.005 (P=0.965). The intelligent diagnostic model based on multiscale deep neural network using wearable electrocardiogram devices has an acceptable accuracy for classifying SVT mechanisms.

Risk prediction models of depression in older adults with chronic diseases

BackgroundDetecting potential depression and identifying the critical predictors of depression among older adults with chronic diseases are essential for timely intervention and management of depression. Therefore, risk prediction models (RPMs) of depression in elderly people should be further explored. MethodsA total of 3959 respondents aged 60 years or over from the wave four survey of the China Health and Retired Longitudinal Study (CHARLS) were included in this study. We used five machine learning (ML) algorithms and three data balancing techniques to construct RPMs of depression and calculated feature importance scores to determine which features are essential to depression. ResultsThe prevalence of depression was 19.2 % among older Chinese adults with chronic diseases in the wave four survey. The random forest (RF) model was more accurate than the other models after balancing the data using the Synthetic Minority Oversampling Technique (SMOTE) algorithm, with an area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC) of 0.957 and 0.920, respectively, a balanced accuracy of 0.891 and a sensitivity of 0.875. Furthermore, we further identified several important predictors between male and female patients via constructed sex-stratified models. LimitationsFurther research on the clinical impact studies of our models and external validation are needed. ConclusionsAfter several techniques were used to address class imbalance issues, most RPMs achieved satisfactory accuracy in predicting depression among elderly people with chronic diseases. RPMs may thus become valuable screening tools for both older individuals and healthcare practitioners to assess the risk of depression.

Stacking Ensemble Learning with Regression Models for Predicting Damage from Terrorist Attacks

Terrorist attacks can cause unexpectedly enormous damage to lives and property. To prevent and mitigate damage from terrorist activities, governments and related organizations must have suitable measures and efficient tools to cope with terrorist attacks. This work proposed a new method based on stacking ensemble learning and regression for predicting damage from terrorist attacks. First, two-layer stacking classifiers were developed and used to indicate if a terrorist attack can cause deaths, injuries, and property damage. For fatal and injury attacks, regression models were utilized to forecast the number of deaths and injuries, respectively. Consequently, the proposed method can efficiently classify casualty terrorist attacks with an average area under precision-recall curves (AUPR) of 0.958. Furthermore, the stacking model can predict property damage attacks with an average AUPR of 0.910. In comparison with existing methods, the proposed method precisely estimates the number of fatalities and injuries with the lowest mean absolute errors of 1.22 and 2.32 for fatal and injury attacks, respectively. According to the superior performance shown, the stacking ensemble models with regression can be utilized as an efficient tool to support emergency prevention and management of terrorist attacks.

SOFA score performs worse than age for predicting mortality in patients with COVID-19.

The use of the Sequential Organ Failure Assessment (SOFA) score, originally developed to describe disease morbidity, is commonly used to predict in-hospital mortality. During the COVID-19 pandemic, many protocols for crisis standards of care used the SOFA score to select patients to be deprioritized due to a low likelihood of survival. A prior study found that age outperformed the SOFA score for mortality prediction in patients with COVID-19, but was limited to a small cohort of intensive care unit (ICU) patients and did not address whether their findings were unique to patients with COVID-19. Moreover, it is not known how well these measures perform across races. In this retrospective study, we compare the performance of age and SOFA score in predicting in-hospital mortality across two cohorts: a cohort of 2,648 consecutive adult patients diagnosed with COVID-19 who were admitted to a large academic health system in the northeastern United States over a 4-month period in 2020 and a cohort of 75,601 patients admitted to one of 335 ICUs in the eICU database between 2014 and 2015. We used age and the maximum SOFA score as predictor variables in separate univariate logistic regression models for in-hospital mortality and calculated area under the receiver operator characteristic curves (AU-ROCs) and area under precision-recall curves (AU-PRCs) for each predictor in both cohorts. Among the COVID-19 cohort, age (AU-ROC 0.795, 95% CI 0.762, 0.828) had a significantly better discrimination than SOFA score (AU-ROC 0.679, 95% CI 0.638, 0.721) for mortality prediction. Conversely, age (AU-ROC 0.628 95% CI 0.608, 0.628) underperformed compared to SOFA score (AU-ROC 0.735, 95% CI 0.726, 0.745) in non-COVID-19 ICU patients in the eICU database. There was no difference between Black and White COVID-19 patients in performance of either age or SOFA Score. Our findings bring into question the utility of SOFA score-based resource allocation in COVID-19 crisis standards of care.

Comparing discriminating abilities of evaluation metrics in link prediction

Link prediction aims to predict the potential existence of links between two unconnected nodes within a network based on the known topological characteristics. Evaluation metrics are used to assess the effectiveness of algorithms in link prediction. The discriminating ability of these evaluation metrics is vitally important for accurately evaluating link prediction algorithms. In this study, we propose an artificial network model, based on which one can adjust a single parameter to monotonically and continuously turn the prediction accuracy of the specifically designed link prediction algorithm. Building upon this foundation, we show a framework to depict the effectiveness of evaluating metrics by focusing on their discriminating ability. Specifically, a quantitative comparison in the abilities of correctly discerning varying prediction accuracies was conducted encompassing nine evaluation metrics: Precision, Recall, F1-Measure, Matthews correlation coefficient, balanced precision, the area under the receiver operating characteristic curve (AUC), the area under the precision-recall curve (AUPR), normalized discounted cumulative gain (NDCG), and the area under the magnified receiver operating characteristic. The results indicate that the discriminating abilities of the three metrics, AUC, AUPR, and NDCG, are significantly higher than those of other metrics.

Hyperparameter tuning using Lévy flight and interactive crossover-based reptile search algorithm for eye movement event classification.

Introduction: Eye movement is one of the cues used in human-machine interface technologies for predicting the intention of users. The developing application in eye movement event detection is the creation of assistive technologies for paralyzed patients. However, developing an effective classifier is one of the main issues in eye movement event detection. Methods: In this paper, bidirectional long short-term memory (BILSTM) is proposed along with hyperparameter tuning for achieving effective eye movement event classification. The Lévy flight and interactive crossover-based reptile search algorithm (LICRSA) is used for optimizing the hyperparameters of BILSTM. The issues related to overfitting are avoided by using fuzzy data augmentation (FDA), and a deep neural network, namely, VGG-19, is used for extracting features from eye movements. Therefore, the optimization of hyperparameters using LICRSA enhances the classification of eye movement events using BILSTM. Results and Discussion: The proposed BILSTM-LICRSA is evaluated by using accuracy, precision, sensitivity, F1-score, area under the receiver operating characteristic (AUROC) curve measure, and area under the precision-recall curve (AUPRC) measure for four datasets, namely, Lund2013, collected dataset, GazeBaseR, and UTMultiView. The gazeNet, human manual classification (HMC), and multi-source information-embedded approach (MSIEA) are used for comparison with the BILSTM-LICRSA. The F1-score of BILSTM-LICRSA for the GazeBaseR dataset is 98.99%, which is higher than that of the MSIEA.

Improving Credit Card Fraud Detection with Data Reduction Approaches

Detecting fraudulent activities in credit card transactions can be challenging due to issues like high dimensionality and class imbalance that are often present in the datasets. To address these challenges, data reduction techniques such as data sampling and feature selection have become essential. In this study, we compare four approaches for data reduction: using data sampling alone, employing feature selection alone, applying data sampling followed by feature selection, and using feature selection followed by data sampling. Additionally, we include results using all features. We build classification models using five Decision Tree-based classifiers and Logistic Regression, and evaluate their performance using two performance metrics: the Area Under the Receiver Operating Characteristic Curve (AUC) and the Area under the Precision–Recall Curve (AUPRC). In this work, we adopt ensemble supervised feature selection (SFS) techniques and Random Undersampling (RUS) for data reduction. The experimental results demonstrate that all four data reduction techniques have the potential to improve the performance of classifiers. These results are valuable since the classifiers available are dependent upon application domains, computing environments, and licensing agreements. However, these techniques can be applied independently of all these dependencies. We recommend utilizing the ensemble SFS followed by RUS (SFS–RUS) approach as the preferred data reduction method due to its ability to run feature selection and data sampling in parallel. Additionally, we find that XGBoost and CatBoost outperform other classifiers.

Identifying potential (re)hemorrhage among sporadic cerebral cavernous malformations using machine learning

The (re)hemorrhage in patients with sporadic cerebral cavernous malformations (CCM) was the primary aim for CCM management. However, accurately identifying the potential (re)hemorrhage among sporadic CCM patients in advance remains a challenge. This study aims to develop machine learning models to detect potential (re)hemorrhage in sporadic CCM patients. This study was based on a dataset of 731 sporadic CCM patients in open data platform Dryad. Sporadic CCM patients were followed up 5 years from January 2003 to December 2018. Support vector machine (SVM), stacked generalization, and extreme gradient boosting (XGBoost) were used to construct models. The performance of models was evaluated by area under receiver operating characteristic curves (AUROC), area under the precision-recall curve (PR-AUC) and other metrics. A total of 517 patients with sporadic CCM were included (330 female [63.8%], mean [SD] age at diagnosis, 42.1 [15.5] years). 76 (re)hemorrhage (14.7%) occurred during follow-up. Among 3 machine learning models, XGBoost model yielded the highest mean (SD) AUROC (0.87 [0.06]) in cross-validation. The top 4 features of XGBoost model were ranked with SHAP (SHapley Additive exPlanations). All-Elements XGBoost model achieved an AUROCs of 0.84 and PR-AUC of 0.49 in testing set, with a sensitivity of 0.86 and a specificity of 0.76. Importantly, 4-Elements XGBoost model developed using top 4 features got a AUROCs of 0.83 and PR-AUC of 0.40, a sensitivity of 0.79, and a specificity of 0.72 in testing set. Two machine learning-based models achieved accurate performance in identifying potential (re)hemorrhages within 5 years in sporadic CCM patients. These models may provide insights for clinical decision-making.

Link prediction in protein–protein interaction network: A similarity multiplied similarity algorithm with paths of length three

Protein–protein interactions (PPIs) are crucial for various biological processes, and predicting PPIs is a major challenge. To solve this issue, the most common method is link prediction. Currently, the link prediction methods based on network Paths of Length Three (L3) have been proven to be highly effective. In this paper, we propose a novel link prediction algorithm, named SMS, which is based on L3 and protein similarities. We first design a mixed similarity that combines the topological structure and attribute features of nodes. Then, we compute the predicted value by summing the product of all similarities along the L3. Furthermore, we propose the Max Similarity Multiplied Similarity (maxSMS) algorithm from the perspective of maximum impact. Our computational prediction results show that on six datasets, including S. cerevisiae, H. sapiens, and others, the maxSMS algorithm improves the precision of the top 500, area under the precision–recall curve, and normalized discounted cumulative gain by an average of 26.99%, 53.67%, and 6.7%, respectively, compared to other optimal methods.

Preoperative prediction model for risk of readmission after total joint replacement surgery: a random forest approach leveraging NLP and unfairness mitigation for improved patient care and cost-effectiveness

BackgroundThe Center for Medicare and Medicaid Services (CMS) imposes payment penalties for readmissions following total joint replacement surgeries. This study focuses on total hip, knee, and shoulder arthroplasty procedures as they account for most joint replacement surgeries. Apart from being a burden to healthcare systems, readmissions are also troublesome for patients. There are several studies which only utilized structured data from Electronic Health Records (EHR) without considering any gender and payor bias adjustments.MethodsFor this study, dataset of 38,581 total knee, hip, and shoulder replacement surgeries performed from 2015 to 2021 at Novant Health was gathered. This data was used to train a random forest machine learning model to predict the combined endpoint of emergency department (ED) visit or unplanned readmissions within 30 days of discharge or discharge to Skilled Nursing Facility (SNF) following the surgery. 98 features of laboratory results, diagnoses, vitals, medications, and utilization history were extracted. A natural language processing (NLP) model finetuned from Clinical BERT was used to generate an NLP risk score feature for each patient based on their clinical notes. To address societal biases, a feature bias analysis was performed in conjunction with propensity score matching. A threshold optimization algorithm from the Fairlearn toolkit was used to mitigate gender and payor biases to promote fairness in predictions.ResultsThe model achieved an Area Under the Receiver Operating characteristic Curve (AUROC) of 0.738 (95% confidence interval, 0.724 to 0.754) and an Area Under the Precision-Recall Curve (AUPRC) of 0.406 (95% confidence interval, 0.384 to 0.433). Considering an outcome prevalence of 16%, these metrics indicate the model’s ability to accurately discriminate between readmission and non-readmission cases within the context of total arthroplasty surgeries while adjusting patient scores in the model to mitigate bias based on patient gender and payor.ConclusionThis work culminated in a model that identifies the most predictive and protective features associated with the combined endpoint. This model serves as a tool to empower healthcare providers to proactively intervene based on these influential factors without introducing bias towards protected patient classes, effectively mitigating the risk of negative outcomes and ultimately improving quality of care regardless of socioeconomic factors.

A nursing note-aware deep neural network for predicting mortality risk after hospital discharge

BackgroundICU readmissions and post-discharge mortality pose significant challenges. Previous studies used EHRs and machine learning models, but mostly focused on structured data. Nursing records contain crucial unstructured information, but their utilization is challenging. Natural language processing (NLP) can extract structured features from clinical text. This study proposes the Crucial Nursing Description Extractor (CNDE) to predict post-ICU discharge mortality rates and identify high-risk patients for unplanned readmission by analyzing electronic nursing records. ObjectiveDeveloped a deep neural network (NurnaNet) with the ability to perceive nursing records, combined with a bio-clinical medicine pre-trained language model (BioClinicalBERT) to analyze the electronic health records (EHRs) in the MIMIC III dataset to predict the death of patients within six month and two year risk. DesignA cohort and system development design was used. Setting(s)Based on data extracted from MIMIC-III, a database of critically ill in the US between 2001 and 2012, the results were analyzed. ParticipantsWe calculated patients' age using admission time and date of birth information from the MIMIC dataset. Patients under 18 or over 89 years old, or who died in the hospital, were excluded. We analyzed 16,973 nursing records from patients' ICU stays. MethodsWe have developed a technology called the Crucial Nursing Description Extractor (CNDE), which extracts key content from text. We use the logarithmic likelihood ratio to extract keywords and combine BioClinicalBERT. We predict the survival of discharged patients after six months and two years and evaluate the performance of the model using precision, recall, the F1-score, the receiver operating characteristic curve (ROC curve), the area under the curve (AUC), and the precision–recall curve (PR curve). ResultsThe research findings indicate that NurnaNet achieved good F1-scores (0.67030, 0.70874) within six months and two years. Compared to using BioClinicalBERT alone, there was an improvement in performance of 2.05 % and 1.08 % for predictions within six months and two years, respectively. ConclusionsCNDE can effectively reduce long-form records and extract key content. NurnaNet has a good F1-score in analyzing the data of nursing records, which helps to identify the risk of death of patients after leaving the hospital and adjust the regular follow-up and treatment plan of relevant medical care as soon as possible.