Highest Area Under The ROC Curve Research Articles

Real-world validation of a framework for automated knowledge driven feature engineering inspired by medical domain experts

Knowledge discovery from real-world data in health care can be demanding due to unstructured data and low registration quality in electronic health records (EHRs). This requires close collaboration of domain experts and data scientists. To perform the knowledge discovery process more effectively and efficiently, a framework for automatic Knowledge Driven Feature Engineering (aKDFE) has been developed. Central to aKDFE is an automated feature engineering (FE), i.e., an automated construction of new, highly informative variables, referred to as features, from those directly observed and recorded, e.g., in EHRs. The framework learns and aggregates domain knowledge to generate features that are more informative compared to those recorded in EHRs or manually engineered (manual FE) as done in medical research projects today.Manual KDFE is a systematic manual FE process, which improves prediction performance without loss of explainability of the predictions. But the following research questions remained open: (i) is it possible to automate KDFE, (ii) are aKDFE features more informative than features from a manual FE process, and (iii) does aKDFE produce explainable and transparent results?To summarize the present study, aKDFE is (i) more efficient than manual FE since it automates the manual knowledge discovery and FE processes. It is (ii) more effective due to its higher predictive power compared to manual KDFE. This was evaluated on a real-world medical research project by comparing the classification ability when using manual and aKDFE features in machine learning (ML) models, measured as the area under the receiver operating characteristic curve (AUROC). The project included 26,992 patients regarding “Negative bone structure effects of antiepileptic drug consumption”. The baseline features (manual FE) used in studied project were compared with features generated by aKDFE; aKDFE-generated features resulted in higher AUROC than baseline features, with a p-value <0.05. Finally, aKDFE (iii) applies and describes data pivoting and feature generation as explicit and transparent operation sequences on EHR features. Inefficiency issues remain, mainly regarding non-automatic FE and baseline generation.Threat of validity to the aKDFE framework exists in the selection of the pivoting method, the generalization of used FE operations and rules, and usage of evaluation metrics.

Development and validation of machine learning algorithms based on electrocardiograms for cardiovascular diagnoses at the population level

Artificial intelligence-enabled electrocardiogram (ECG) algorithms are gaining prominence for the early detection of cardiovascular (CV) conditions, including those not traditionally associated with conventional ECG measures or expert interpretation. This study develops and validates such models for simultaneous prediction of 15 different common CV diagnoses at the population level. We conducted a retrospective study that included 1,605,268 ECGs of 244,077 adult patients presenting to 84 emergency departments or hospitals, who underwent at least one 12-lead ECG from February 2007 to April 2020 in Alberta, Canada, and considered 15 CV diagnoses, as identified by International Classification of Diseases, 10th revision (ICD-10) codes: atrial fibrillation (AF), supraventricular tachycardia (SVT), ventricular tachycardia (VT), cardiac arrest (CA), atrioventricular block (AVB), unstable angina (UA), ST-elevation myocardial infarction (STEMI), non-STEMI (NSTEMI), pulmonary embolism (PE), hypertrophic cardiomyopathy (HCM), aortic stenosis (AS), mitral valve prolapse (MVP), mitral valve stenosis (MS), pulmonary hypertension (PHTN), and heart failure (HF). We employed ResNet-based deep learning (DL) using ECG tracings and extreme gradient boosting (XGB) using ECG measurements. When evaluated on the first ECGs per episode of 97,631 holdout patients, the DL models had an area under the receiver operating characteristic curve (AUROC) of <80% for 3 CV conditions (PTE, SVT, UA), 80–90% for 8 CV conditions (CA, NSTEMI, VT, MVP, PHTN, AS, AF, HF) and an AUROC > 90% for 4 diagnoses (AVB, HCM, MS, STEMI). DL models outperformed XGB models with about 5% higher AUROC on average. Overall, ECG-based prediction models demonstrated good-to-excellent prediction performance in diagnosing common CV conditions.

Development and Validation of a Prediction Model for Acute Hypotensive Events in Intensive Care Unit Patients.

Background: Persistent hypotension in the intensive care unit (ICU) is associated with increased mortality. Predicting acute hypotensive events can lead to timely intervention. We aimed to develop a prediction model of acute hypotensive events in patients admitted to the ICU. Methods: We included adult patients admitted to the Nagoya City University (NCU) Hospital ICU between January 2018 and December 2021 for model training and internal validation. The MIMIC-III database was used for external validation. A hypotensive event was defined as a mean arterial pressure < 60 mmHg for at least 5 min in 10 min. The input features were age, sex, and time-series data for vital signs. We compared the area under the receiver-operating characteristic curve (AUROC) of three machine-learning algorithms: logistic regression, the light gradient boosting machine (LightGBM), and long short-term memory (LSTM). Results: Acute hypotensive events were found in 1325/1777 (74.6%) and 2691/5266 (51.1%) of admissions in the NCU and MIMIC-III cohorts, respectively. In the internal validation, the LightGBM model had the highest AUROC (0.835), followed by the LSTM (AUROC 0.834) and logistic regression (AUROC 0.821) models. Applying only blood pressure-related features, the LSTM model achieved the highest AUROC (0.843) and consistently showed similar results in external and internal validation. Conclusions: The LSTM model using only blood pressure-related features had the highest AUROC with comparable performance in external validation.

Automated annotation of disease subtypes

BackgroundDistinguishing diseases into distinct subtypes is crucial for study and effective treatment strategies. The Open Targets Platform (OT) integrates biomedical, genetic, and biochemical datasets to empower disease ontologies, classifications, and potential gene targets. Nevertheless, many disease annotations are incomplete, requiring laborious expert medical input. This challenge is especially pronounced for rare and orphan diseases, where resources are scarce. MethodsWe present a machine learning approach to identifying diseases with potential subtypes, using the approximately 23,000 diseases documented in OT. We derive novel features for predicting diseases with subtypes using direct evidence. Machine learning models were applied to analyze feature importance and evaluate predictive performance for discovering both known and novel disease subtypes. ResultsOur model achieves a high (89.4%) ROC AUC (Area Under the Receiver Operating Characteristic Curve) in identifying known disease subtypes. We integrated pre-trained deep-learning language models and showed their benefits. Moreover, we identify 515 disease candidates predicted to possess previously unannotated subtypes. ConclusionsOur models can partition diseases into distinct subtypes. This methodology enables a robust, scalable approach for improving knowledge-based annotations and a comprehensive assessment of disease ontology tiers. Our candidates are attractive targets for further study and personalized medicine, potentially aiding in the unveiling of new therapeutic indications for sought-after targets.

BiMPADR: A Deep Learning Framework for Predicting Adverse Drug Reactions in New Drugs.

Detecting the unintended adverse reactions of drugs (ADRs) is a crucial concern in pharmacological research. The experimental validation of drug-ADR associations often entails expensive and time-consuming investigations. Thus, a computational model to predict ADRs from known associations is essential for enhanced efficiency and cost-effectiveness. Here, we propose BiMPADR, a novel model that integrates drug gene expression into adverse reaction features using a message passing neural network on a bipartite graph of drugs and adverse reactions, leveraging publicly available data. By combining the computed adverse reaction features with the structural fingerprints of drugs, we predict the association between drugs and adverse reactions. Our models obtained high AUC (area under the receiver operating characteristic curve) values ranging from 0.861 to 0.907 in an external drug validation dataset under differential experiment conditions. The case study on multiple BET inhibitors also demonstrated the high accuracy of our predictions, and our model's exploration of potential adverse reactions for HWD-870 has contributed to its research and development for market approval. In summary, our method would provide a promising tool for ADR prediction and drug safety assessment in drug discovery and development.

Validation of three health literacy screening questions compared with S-TOFHLA in a low-income diverse English- and Spanish-Speaking population.

Clinicians need a tool to gauge patients' ability to understand health conditions and treatment options. The Short-form Test of Functional Health Literacy in Adults (S-TOFHLA) is the gold standard for this, but its length is prohibitive for use in clinical settings. This study seeks to validate a novel three-item question set for predicting health literacy. This cross-sectional study utilized an in-person questionnaire alongside the S-TOFHLA. The sample included 2027 English- and Spanish-speaking adults (≥18years) recruited from primary care practices serving a low-income eastern Pennsylvania community. Most patients (57.7%) identified as Hispanic. Diagnostic accuracy of each question and aggregated scores were assessed against the validated survey by calculating the area under the receiver operating characteristic (AUROC) curve. Questions in the 'Problems Learning' and 'Help Reading' domains (AUROC 0.66 for each) performed better than the 'Confident Forms' question (AUROC 0.64). Summing all three scores resulted in an even higher AUROC curve (0.71). Cronbach's alpha of the combined items was 0.696. Study results suggest that any of the three questions are viable options for screening health literacy levels of diverse patients in primary care clinical settings. However, they perform better as a summed score than when used individually.

Artificial intelligence for ultrasound microflow imaging in breast cancer diagnosis.

To develop and evaluate artificial intelligence (AI) algorithms for ultrasound (US) microflow imaging (MFI) in breast cancer diagnosis. We retrospectively collected a dataset consisting of 516 breast lesions (364 benign and 152 malignant) in 471 women who underwent B-mode US and MFI. The internal dataset was split into training (n = 410) and test datasets (n = 106) for developing AI algorithms from deep convolutional neural networks from MFI. AI algorithms were trained to provide malignancy risk (0-100%). The developed AI algorithms were further validated with an independent external dataset of 264 lesions (229 benign and 35 malignant). The diagnostic performance of B-mode US, AI algorithms, or their combinations was evaluated by calculating the area under the receiver operating characteristic curve (AUROC). The AUROC of the developed three AI algorithms (0.955-0.966) was higher than that of B-mode US (0.842, P < 0.0001). The AUROC of the AI algorithms on the external validation dataset (0.892-0.920) was similar to that of the test dataset. Among the AI algorithms, no significant difference was found in all performance metrics combined with or without B-mode US. Combined B-mode US and AI algorithms had a higher AUROC (0.963-0.972) than that of B-mode US (P < 0.0001). Combining B-mode US and AI algorithms significantly decreased the false-positive rate of BI-RADS category 4A lesions from 87% to 13% (P < 0.0001). AI-based MFI diagnosed breast cancers with better performance than B-mode US, eliminating 74% of false-positive diagnoses in BI-RADS category 4A lesions.

Abstract LB243: Deep learning-based molecular characterization of lung cancers from never smokers using hematoxylin and eosin-stained whole slide images

Abstract Purpose: This study employs cutting-edge deep learning techniques to comprehensively analyze hematoxylin and eosin-stained (H&amp;E) whole slide images (WSIs) to predict which tumors carry driver gene alterations, mutational signatures, and additional molecular features among lung cancers from never-smokers (LCINS)— the seventh leading cause of cancer death worldwide. Method: A total of 464 H&amp;E-stained WSIs of lung adenocarcinomas from the Sherlock-Lung study were utilized, with 325 WSIs for model training and 139 for testing. Alongside the WSIs, genomic data, including mutations (overall nonsynonymous mutations and driver mutations only), fusions, or copy numbers in driver genes (TP53, KRAS, EGFR, CDKN2A, MDM2, ALK, RBM10), mutational signatures (APOBEC), molecular features (Kataegies, Whole Genome Doubling (WGD) status, Tumor Mutational Burden (TMB)), and specific hotspot driver mutation in EGFR (p.L858R and p.E746_A750del) and KRAS (p.G12C, p.G12V, and p.G12D), were included for all tumors. Each WSI was assigned different labels based on the presence or absence of these genomic alterations or features. We used a customized multilabel binary deep learning model based on ResNet50, a residual convolutional neural network, for analyzing the data. The network was trained from scratch, with an initial epoch set to 100. Results: Our methodology demonstrated high predictive performance, measured by the area under the receiver operating characteristic (AUROC). In the presence or absence category, we achieved high AUROC for detecting any alterations in these driver genes: TP53 (0.81), KRAS (0.92), EGFR (0.93), CDKN2A deletion (0.88), MDM2 amplification (0.94), and ALK fusion (0.86). We also evaluated molecular features, obtaining AUROC scores of 0.79 for WGD status and 0.86 for Kataegies. Moderate AUROC scores were observed for tumors with RBM10 mutations (0.67), high TMB (0.64), and APOBEC signatures (0.56). When focusing on driver mutations only, we achieved high AUROC scores for EGFR (0.88), KRAS (0.84), TP53 (0.93), and RBM10 (0.78). In addition, our model successfully predicted tumors with specific mutations in EGFR (p.L858R = 0.79; p.E746_A750del = 0.77) and KRAS (p.G12C = 0.77). Performance was suboptimal for KRAS p.G12V (0.41) and KRAS p.G12D (0.47). Conclusions: Our deep learning network achieves high prediction scores in identifying tumors with critical driver gene alterations and actionable mutations, holding promise for potential clinical use. In the future, the model could be optimized as a screening assay to guide molecular testing and therapeutic management of patients with LCINS. Citation Format: Monjoy Saha, Tongwu Zhang, Praphulla Bhawsar, Wei Zhao, Jianxin Shi, Soo Ryum Yang, Jonas Almeida, Maria Teresa Landi. Deep learning-based molecular characterization of lung cancers from never smokers using hematoxylin and eosin-stained whole slide images [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 2 (Late-Breaking, Clinical Trial, and Invited Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(7_Suppl):Abstract nr LB243.

Predicting COVID-19 Vaccination Uptake Using a Small and Interpretable Set of Judgment and Demographic Variables: Cross-Sectional Cognitive Science Study.

Despite COVID-19 vaccine mandates, many chose to forgo vaccination, raising questions about the psychology underlying how judgment affects these choices. Research shows that reward and aversion judgments are important for vaccination choice; however, no studies have integrated such cognitive science with machine learning to predict COVID-19 vaccine uptake. This study aims to determine the predictive power of a small but interpretable set of judgment variables using 3 machine learning algorithms to predict COVID-19 vaccine uptake and interpret what profile of judgment variables was important for prediction. We surveyed 3476 adults across the United States in December 2021. Participants answered demographic, COVID-19 vaccine uptake (ie, whether participants were fully vaccinated), and COVID-19 precaution questions. Participants also completed a picture-rating task using images from the International Affective Picture System. Images were rated on a Likert-type scale to calibrate the degree of liking and disliking. Ratings were computationally modeled using relative preference theory to produce a set of graphs for each participant (minimum R2>0.8). In total, 15 judgment features were extracted from these graphs, 2 being analogous to risk and loss aversion from behavioral economics. These judgment variables, along with demographics, were compared between those who were fully vaccinated and those who were not. In total, 3 machine learning approaches (random forest, balanced random forest [BRF], and logistic regression) were used to test how well judgment, demographic, and COVID-19 precaution variables predicted vaccine uptake. Mediation and moderation were implemented to assess statistical mechanisms underlying successful prediction. Age, income, marital status, employment status, ethnicity, educational level, and sex differed by vaccine uptake (Wilcoxon rank sum and chi-square P<.001). Most judgment variables also differed by vaccine uptake (Wilcoxon rank sum P<.05). A similar area under the receiver operating characteristic curve (AUROC) was achieved by the 3 machine learning frameworks, although random forest and logistic regression produced specificities between 30% and 38% (vs 74.2% for BRF), indicating a lower performance in predicting unvaccinated participants. BRF achieved high precision (87.8%) and AUROC (79%) with moderate to high accuracy (70.8%) and balanced recall (69.6%) and specificity (74.2%). It should be noted that, for BRF, the negative predictive value was <50% despite good specificity. For BRF and random forest, 63% to 75% of the feature importance came from the 15 judgment variables. Furthermore, age, income, and educational level mediated relationships between judgment variables and vaccine uptake. The findings demonstrate the underlying importance of judgment variables for vaccine choice and uptake, suggesting that vaccine education and messaging might target varying judgment profiles to improve uptake. These methods could also be used to aid vaccine rollouts and health care preparedness by providing location-specific details (eg, identifying areas that may experience low vaccination and high hospitalization).

Real-time machine learning model to predict short-term mortality in critically ill patients: development and international validation

BackgroundA real-time model for predicting short-term mortality in critically ill patients is needed to identify patients at imminent risk. However, the performance of the model needs to be validated in various clinical settings and ethnicities before its clinical application. In this study, we aim to develop an ensemble machine learning model using routinely measured clinical variables at a single academic institution in South Korea.MethodsWe developed an ensemble model using deep learning and light gradient boosting machine models. Internal validation was performed using the last two years of the internal cohort dataset, collected from a single academic hospital in South Korea between 2007 and 2021. External validation was performed using the full Medical Information Mart for Intensive Care (MIMIC), eICU-Collaborative Research Database (eICU-CRD), and Amsterdam University Medical Center database (AmsterdamUMCdb) data. The area under the receiver operating characteristic curve (AUROC) was calculated and compared to that for the National Early Warning Score (NEWS).ResultsThe developed model (iMORS) demonstrated high predictive performance with an internal AUROC of 0.964 (95% confidence interval [CI] 0.963–0.965) and external AUROCs of 0.890 (95% CI 0.889–0.891) for MIMIC, 0.886 (95% CI 0.885–0.887) for eICU-CRD, and 0.870 (95% CI 0.868–0.873) for AmsterdamUMCdb. The model outperformed the NEWS with higher AUROCs in the internal and external validation (0.866 for the internal, 0.746 for MIMIC, 0.798 for eICU-CRD, and 0.819 for AmsterdamUMCdb; p < 0.001).ConclusionsOur real-time machine learning model to predict short-term mortality in critically ill patients showed excellent performance in both internal and external validations. This model could be a useful decision-support tool in the intensive care units to assist clinicians.

Computed tomography-based body composition parameters can predict short-term prognosis in ulcerative colitis patients.

Emerging evidence suggests a potential relationship between body composition and short-term prognosis of ulcerative colitis (UC). Early and accurate assessment of rapid remission based on conventional therapy via abdominal computed tomography (CT) images has rarely been investigated. This study aimed to build a prediction model using CT-based body composition parameters for UC risk stratification. In total, 138 patients with abdominal CT images were enrolled. Eleven quantitative parameters related to body composition involving skeletal muscle mass, visceral adipose tissue (VAT), and subcutaneous adipose tissue (SAT) were measured and calculated using a semi-automated segmentation method. A prediction model was established with significant parameters using a multivariable logistic regression. The receiver operating characteristic (ROC) curve was plotted to evaluate prediction performance. Subgroup analyses were implemented to evaluate the diagnostic efficiency of the prediction model between different disease locations, centers, and CT scanners. The Delong test was used for statistical comparison of ROC curves. VAT density, SAT density, gender, and visceral obesity were significantly statistically different between remission and invalidation groups (all p < 0.05). The accuracy, sensitivity, specificity, and area under the ROC curve (AUC) of the prediction model were 82.61%, 95.45%, 69.89%, and 0.855 (0.792-0.917), respectively. The positive predictive value and negative predictive value were 70.79% and 93.88%, respectively. No significant differences in the AUC of the prediction model were found in different subgroups (all p > 0.05). The predicting model constructed with CT-based body composition parameters is a potential non-invasive approach for short-term prognosis identification and risk stratification. Additionally, VAT density was an independent predictor for escalating therapeutic regimens in UC cohorts. The CT images were used for evaluating body composition and risk stratification of ulcerative colitis patients, and a potential non-invasive prediction model was constructed to identify non-responders with conventional therapy for making therapeutic regimens timely and accurately. • CT-based prediction models help divide patients into invalidation and remission groups in UC. • Results of the subgroup analysis confirmed the stability of the prediction model with a high AUC (all > 0.820). • The visceral adipose tissue density was an independent predictor of bad short-term prognosis in UC.

The utility of quantifying the orientation of breast masses in ultrasound imaging.

The aim of this study was to quantify the orientation of breast masses and determine whether it can enhance the utility of a not parallel orientation in predicting breast mass malignancy. A total of 15,746 subjects who underwent breast ultrasound examinations were initially enrolled in the study. Further evaluation was performed on subjects with solid breast masses (≤ 5cm) intended for surgical resection and/or biopsy. The orientation angle, defined as the acute angle between the align of the maximal longitudinal diameter of the breast mass and the surface of the breast skin, was measured. Receiver operating characteristic (ROC) curve analysis was conducted, and various performance measures including sensitivity, specificity, positive and negative predictive values, accuracy, odds ratio, and the area under the ROC curve (AUC) were calculated. Multivariate analysis was performed to determine if the orientation angle was an independent predictor of breast malignancy. Decision curve analysis (DCA) was also conducted to assess the net benefit of adopting the orientation angle for predicting breast mass malignancy. The final analysis included 83 subjects with breast cancer and 135 subjects with benign masses. The intra-group correlation coefficient for the measurement of the orientation angle of breast masses was 0.986 (P = 0.001), indicating high reproducibility. The orientation angles of malignant and benign breast masses were 36.51 ± 14.90 (range: 10.7-88.6) degrees and 15.28 ± 8.40 (range: 0.0-58.7) degrees, respectively, and there was a significant difference between them (P < 0.001). The cutoff value for the orientation angle was determined to be 22.9°. The sensitivity, specificity, positive and negative predictive values, accuracy, odds ratio, and AUC for the prediction of breast malignancy using the orientation angle were 88.0%, 87.4%, 81.1%, 92.2%, 87.6%, 50.67%, and 0.925%, respectively. Multivariate analysis revealed that the orientation angle (> 22.9°), not circumscribed margin, and calcifications of the breast mass were independent factors predicting breast malignancy. The net benefit of adopting the orientation angle for predicting breast malignancy was 0.303. Based on these findings, it can be concluded that quantifying the orientation angle of breast masses is useful in predicting breast malignancy, as it demonstrates high sensitivity, specificity, AUC, and standardized net benefit. It optimizes the utility of the not parallel orientation in assessing breast mass malignancy.

A comparative study of explainable ensemble learning and logistic regression for predicting in-hospital mortality in the emergency department

This study addresses the challenges associated with emergency department (ED) overcrowding and emphasizes the need for efficient risk stratification tools to identify high-risk patients for early intervention. While several scoring systems, often based on logistic regression (LR) models, have been proposed to indicate patient illness severity, this study aims to compare the predictive performance of ensemble learning (EL) models with LR for in-hospital mortality in the ED. A cross-sectional single-center study was conducted at the ED of Imam Reza Hospital in northeast Iran from March 2016 to March 2017. The study included adult patients with one to three levels of emergency severity index. EL models using Bagging, AdaBoost, random forests (RF), Stacking and extreme gradient boosting (XGB) algorithms, along with an LR model, were constructed. The training and validation visits from the ED were randomly divided into 80% and 20%, respectively. After training the proposed models using tenfold cross-validation, their predictive performance was evaluated. Model performance was compared using the Brier score (BS), The area under the receiver operating characteristics curve (AUROC), The area and precision–recall curve (AUCPR), Hosmer–Lemeshow (H–L) goodness-of-fit test, precision, sensitivity, accuracy, F1-score, and Matthews correlation coefficient (MCC). The study included 2025 unique patients admitted to the hospital’s ED, with a total percentage of hospital deaths at approximately 19%. In the training group and the validation group, 274 of 1476 (18.6%) and 152 of 728 (20.8%) patients died during hospitalization, respectively. According to the evaluation of the presented framework, EL models, particularly Bagging, predicted in-hospital mortality with the highest AUROC (0.839, CI (0.802–0.875)) and AUCPR = 0.64 comparable in terms of discrimination power with LR (AUROC (0.826, CI (0.787–0.864)) and AUCPR = 0.61). XGB achieved the highest precision (0.83), sensitivity (0.831), accuracy (0.842), F1-score (0.833), and the highest MCC (0.48). Additionally, the most accurate models in the unbalanced dataset belonged to RF with the lowest BS (0.128). Although all studied models overestimate mortality risk and have insufficient calibration (P > 0.05), stacking demonstrated relatively good agreement between predicted and actual mortality. EL models are not superior to LR in predicting in-hospital mortality in the ED. Both EL and LR models can be considered as screening tools to identify patients at risk of mortality.

Validation of genetic risk scores for coronary artery disease, developed on European population samples, in Russian population

Aim. To evaluate the information content of genetic risk scores (GRSs) for coronary artery disease (CAD), previously developed on European populations, in representatives of the Russian population.Material and methods. The work involved 1685 people from the ESSE-Ivanovo epidemiological study. CAD was verified in 3,1% of individuals. The coronary composite endpoint was assessed annually during 8-year follow-up. Next generation sequencing was performed using a targeted panel. Logistic regression analysis and area under the ROC curve (AUC) were used. Age, sex, and smoking status were taken into account in the multivariate model.Results. Of the 16 GRSs included in the analysis, only 2 GRSs demonstrated significance in the univariate analysis of association with CAD (highest AUC — 0,577). In a multivariate model, with an increase by 1 standard deviation (SD) for the 6 studied GRSs, a significant association with CAD was obtained — the odds ratio varied in the range of 1,31-1,47. The two GRSs demonstrated significant differences in the incidence of CAD between the groups corresponding to the upper and lower quintiles. Forty-five endpoints were registered. The risk ratio for the end point with an increase in GRS by 1 SD, taking into account cofactors, exceeded statistical significance for the 9 analyzed GRS and was in the range of 1,36-1,54.Conclusion. For the first time in Russia, 16 CAD GRSs, previously developed on European samples, was validated. The results were reproduced only for a few of the studied CAD SGRs.

A Deep Learning Framework for Predicting Patient Decannulation on Extracorporeal Membrane Oxygenation Devices: Development and Model Analysis Study.

Venovenous extracorporeal membrane oxygenation (VV-ECMO) is a therapy for patients with refractory respiratory failure. The decision to decannulate someone from extracorporeal membrane oxygenation (ECMO) often involves weaning trials and clinical intuition. To date, there are limited prognostication metrics to guide clinical decision-making to determine which patients will be successfully weaned and decannulated. This study aims to assist clinicians with the decision to decannulate a patient from ECMO, using Continuous Evaluation of VV-ECMO Outcomes (CEVVO), a deep learning-based model for predicting success of decannulation in patients supported on VV-ECMO. The running metric may be applied daily to categorize patients into high-risk and low-risk groups. Using these data, providers may consider initiating a weaning trial based on their expertise and CEVVO. Data were collected from 118 patients supported with VV-ECMO at the Columbia University Irving Medical Center. Using a long short-term memory-based network, CEVVO is the first model capable of integrating discrete clinical information with continuous data collected from an ECMO device. A total of 12 sets of 5-fold cross validations were conducted to assess the performance, which was measured using the area under the receiver operating characteristic curve (AUROC) and average precision (AP). To translate the predicted values into a clinically useful metric, the model results were calibrated and stratified into risk groups, ranging from 0 (high risk) to 3 (low risk). To further investigate the performance edge of CEVVO, 2 synthetic data sets were generated using Gaussian process regression. The first data set preserved the long-term dependency of the patient data set, whereas the second did not. CEVVO demonstrated consistently superior classification performance compared with contemporary models (P<.001 and P=.04 compared with the next highest AUROC and AP). Although the model's patient-by-patient predictive power may be too low to be integrated into a clinical setting (AUROC 95% CI 0.6822-0.7055; AP 95% CI 0.8515-0.8682), the patient risk classification system displayed greater potential. When measured at 72 hours, the high-risk group had a successful decannulation rate of 58% (7/12), whereas the low-risk group had a successful decannulation rate of 92% (11/12; P=.04). When measured at 96 hours, the high- and low-risk groups had a successful decannulation rate of 54% (6/11) and 100% (9/9), respectively (P=.01). We hypothesized that the improved performance of CEVVO was owing to its ability to efficiently capture transient temporal patterns. Indeed, CEVVO exhibited improved performance on synthetic data with inherent temporal dependencies (P<.001) compared with logistic regression and a dense neural network. The ability to interpret and integrate large data sets is paramount for creating accurate models capable of assisting clinicians in risk stratifying patients supported on VV-ECMO. Our framework may guide future incorporation of CEVVO into more comprehensive intensive care monitoring systems.

Differentiating IDH-mutant astrocytomas and 1p19q-codeleted oligodendrogliomas using DSC-PWI: high performance through cerebral blood volume and percentage of signal recovery percentiles.

Presurgical differentiation between astrocytomas and oligodendrogliomas remains an unresolved challenge in neuro-oncology. This research aims to provide a comprehensive understanding of each tumor's DSC-PWI signatures, evaluate the discriminative capacity of cerebral blood volume (CBV) and percentage of signal recovery (PSR) percentile values, and explore the synergy of CBV and PSR combination for pre-surgical differentiation. Patients diagnosed with grade 2 and 3 IDH-mutant astrocytomas and IDH-mutant 1p19q-codeleted oligodendrogliomas were retrospectively retrieved (2010-2022). 3D segmentations of each tumor were conducted, and voxel-level CBV and PSR were extracted to compute mean, minimum, maximum, and percentile values. Statistical comparisons were performed using the Mann-Whitney U test and the area under the receiver operating characteristic curve (AUC-ROC). Lastly, the five most discriminative variables were combined for classification with internal cross-validation. The study enrolled 52 patients (mean age 45-year-old, 28 men): 28 astrocytomas and 24 oligodendrogliomas. Oligodendrogliomas exhibited higher CBV and lower PSR than astrocytomas across all metrics (e.g., mean CBV = 2.05 and 1.55, PSR = 0.68 and 0.81 respectively). The highest AUC-ROCs and the smallest p values originated from CBV and PSR percentiles (e.g., PSRp70 AUC-ROC = 0.84 and p value = 0.0005, CBVp75 AUC-ROC = 0.8 and p value = 0.0006). The mean, minimum, and maximum values yielded lower results. Combining the best five variables (PSRp65, CBVp70, PSRp60, CBVp75, and PSRp40) achieved a mean AUC-ROC of 0.87 for differentiation. Oligodendrogliomas exhibit higher CBV and lower PSR than astrocytomas, traits that are emphasized when considering percentiles rather than mean or extreme values. The combination of CBV and PSR percentiles results in promising classification outcomes. The combination of histogram-derived percentile values of cerebral blood volume and percentage of signal recovery from DSC-PWI enhances the presurgical differentiation between astrocytomas and oligodendrogliomas, suggesting that incorporating these metrics into clinical practice could be beneficial. • The unsupervised selection of percentile values for cerebral blood volume and percentage of signal recovery enhances presurgical differentiation of astrocytomas and oligodendrogliomas. • Oligodendrogliomas exhibit higher cerebral blood volume and lower percentage of signal recovery than astrocytomas. • Cerebral blood volume and percentage of signal recovery combined provide a broader perspective on tumor vasculature and yield promising results for this preoperative classification.

Potential role of Fibrosis-4 score in hepatocellular carcinoma screening: The Kangbuk Samsung Health Study.

Hepatocellular carcinoma (HCC) is a major cause of cancer-related death, with low survival rates worldwide. Fatty liver disease (FLD) significantly contributes to HCC. We studied the screening performance of different methods for identifying HCC in patients with FLD or with metabolic risk factors for FLD. Korean adults (n=340825) without a prior HCC diagnosis were categorized into four groups: normal (G1), ≥2 metabolic risk factors (G2), FLD (G3), and viral liver disease or liver cirrhosis (G4). The National Cancer Registry data were used to identify HCC cases within 12months. We assessed the area under the receiver operating characteristic curve, sensitivity, specificity, and positive and negative predictive values of individual or combined screening methods. In 93 HCC cases, 71 were identified in G4, whereas 20 cases (21.5%) in G2 and G3 combined where ultrasound and Fibrosis-4 performed similarly to alpha-fetoprotein and ultrasound. In G2, Fibrosis-4 and ultrasound had the highest area under the receiver operating characteristic curve (0.93 [0.87-0.99]), whereas in G3, the combined screening methods had the highest area under the receiver operating characteristic curve (0.98 [0.95-1.00]). The positive predictive value was lower in G2 and G3 than in G4, but was >5% when restricted to a high Fibrosis-4 score. More than 21% of HCC cases were observed in patients with diagnosed FLD or at risk of FLD with metabolic risk factors. Nevertheless, screening for HCC in individuals without cirrhosis or viral hepatitis yielded very low results, despite the potential value of the Fibrosis-4 score in identifying individuals at high risk of HCC.

Impact of random oversampling and random undersampling on the performance of prediction models developed using observational health data

BackgroundThere is currently no consensus on the impact of class imbalance methods on the performance of clinical prediction models. We aimed to empirically investigate the impact of random oversampling and random undersampling, two commonly used class imbalance methods, on the internal and external validation performance of prediction models developed using observational health data.MethodsWe developed and externally validated prediction models for various outcomes of interest within a target population of people with pharmaceutically treated depression across four large observational health databases. We used three different classifiers (lasso logistic regression, random forest, XGBoost) and varied the target imbalance ratio. We evaluated the impact on model performance in terms of discrimination and calibration. Discrimination was assessed using the area under the receiver operating characteristic curve (AUROC) and calibration was assessed using calibration plots.ResultsWe developed and externally validated a total of 1,566 prediction models. On internal and external validation, random oversampling and random undersampling generally did not result in higher AUROCs. Moreover, we found overestimated risks, although this miscalibration could largely be corrected by recalibrating the models towards the imbalance ratios in the original dataset.ConclusionsOverall, we found that random oversampling or random undersampling generally does not improve the internal and external validation performance of prediction models developed in large observational health databases. Based on our findings, we do not recommend applying random oversampling or random undersampling when developing prediction models in large observational health databases.

The usefulness of lactate/albumin ratio, C-reactive protein/albumin ratio, procalcitonin/albumin ratio, SOFA, and qSOFA in predicting the prognosis of patients with sepsis who presented to EDs

PurposeEarly identification of sepsis with a poor prognosis in the emergency department (ED) is crucial for prompt management and improved outcomes. This study aimed to examine the predictive value of sequential organ failure assessment (SOFA), quick SOFA (qSOFA), lactate to albumin ratio (LAR), C-reactive protein to albumin ratio (CAR), and procalcitonin to albumin ratio (PAR), obtained in the ED, as predictors for 28-day mortality in patients with sepsis and septic shock. Materials and methodsWe included 3499 patients (aged ≥19 years) from multicenter registry of the Korean Shock Society between October 2015 and December 2019. The SOFA score, qSOFA score, and lactate level at the time of registry enrollment were used. Albumin, C-reactive protein, and procalcitonin levels were obtained from the initial laboratory results measured upon ED arrival. We evaluated the predictive accuracy for 28-day mortality using the area under the receiver operating characteristic (AUROC) curve. A multivariable logistic regression analysis of the independent predictors of 28-day mortality was performed. The SOFA score, LAR, CAR, and PAR were converted to categorical variables using Youden's index and analyzed. Adjusting for confounding factors such as age, sex, comorbidities, and infection focus, adjusted odds ratios (aOR) were calculated. ResultsOf the 3499 patients, 2707 (77.4%) were survivors, whereas 792 (22.6%) were non-survivors. The median age of the patients was 70 (25th–75th percentiles, 61–78), and 2042 (58.4%) were male. LAR for predicting 28-day mortality had the highest AUROC, followed by the SOFA score (0.715; 95% confidence interval (CI): 0.69–0.74 and 0.669; 95% CI: 0.65–0.69, respectively). The multivariable logistic regression analysis revealed that the aOR of LAR >1.52 was 3.75 (95% CI: 3.16–4.45), and the aOR, of SOFA score at enrollment >7.5 was 2.67 (95% CI: 2.25–3.17). ConclusionThe results of this study showed that LAR is a relatively strong predictor of sepsis prognosis in the ED setting, indicating its potential as a straightforward and practical prognostic factor. This finding may assist healthcare providers in the ED by providing them with tools to risk-stratify patients and predict their mortality.

STREM-1: A Biomarker of Mortality in Severe Malaria Impacted by Acute Kidney Injury.

Malaria is an important cause of mortality in African children. Identification of biomarkers to identify children at risk of mortality has the potential to improve outcomes. We evaluated 11 biomarkers of host response in 592 children with severe malaria. The primary outcome was biomarker performance for predicting mortality. Biomarkers were evaluated using receiver operating characteristic (ROC) curve analysis comparing the area under the ROC curve (AUROC). Mortality was 7.3% among children in the study with 72% of deaths occurring within 24 hours of admission. Among the candidate biomarkers, soluble triggering receptor expressed on myeloid cells 1 (sTREM-1) had the highest AUROC (0.78 [95% confidence interval, .70-.86]), outperforming several other biomarkers including C-reactive protein and procalcitonin. sTREM-1 was the top-performing biomarker across prespecified subgroups (malaria definition, site, sex, nutritional status, age). Using established cutoffs, we evaluated mortality across sTREM-1 risk zones. Among children with acute kidney injury, 39.9% of children with a critical-risk sTREM-1 result had an indication for dialysis. When evaluated relative to a disease severity score, sTREM-1 improved mortality prediction (difference in AUROC, P = .016). sTREM-1 is a promising biomarker to guide rational allocation of clinical resources and should be integrated into clinical decision support algorithms, particularly when acute kidney injury is suspected.