Holdout Test Dataset Research Articles

Machine learning and explainable artificial intelligence to predict pathologic stage in men with localized prostate cancer.

Though several nomograms exist, machine learning (ML) approaches might improve prediction of pathologic stage in patients with prostate cancer. To develop ML models to predict pathologic stage that outperform existing nomograms that use readily available clinicopathologic variables. Patients with prostate adenocarcinoma who underwent surgery were identified in the National Cancer Database. Seven ML models were trained to predict organ-confined (OC) disease, extracapsular extension, seminal vesicle invasion (SVI), and lymph node involvement (LNI). Model performance was measured using area under the curve (AUC) on a holdout testing data set. Clinical utility was evaluated using decision curve analysis (DCA). Performance metrics were confirmed on an external validation data set. The ML-based extreme gradient boosted trees model achieved the best performance with an AUC of 0.744, 0.749, 0.816, 0.811 for the OC, ECE, SVI, and LNI models, respectively. The MSK nomograms achieved an AUC of 0.708, 0.742, 0.806, 0.802 for the OC, ECE, SVI, and LNI models, respectively. These models also performed the best on DCA. Findings were consistent on both a holdout internal validation data set as well as an external validation data set. Our ML models better predicted pathologic stage relative to existing nomograms at predicting pathologic stage. Accurate prediction of pathologic stage can help oncologists and patients determine optimal definitive treatment options for patients with prostate cancer.

Shape matters: unsupervised exploration of IDH-wildtype glioma imaging survival predictors

Abstract Objectives This study examines clustering based on shape radiomic features and tumor volume to identify IDH-wildtype glioma phenotypes and assess their impact on overall survival (OS). Materials and methods This retrospective study included 436 consecutive patients diagnosed with IDH-wt glioma who underwent preoperative MR imaging. Alongside the total tumor volume, nine distinct shape radiomic features were extracted using the PyRadiomics framework. Different imaging phenotypes were identified using partition around medoids (PAM) clustering on the training dataset (348/436). The prognostic efficacy of these phenotypes in predicting OS was evaluated on the test dataset (88/436). External validation was performed using the public UCSF glioma dataset (n = 397). A decision-tree algorithm was employed to determine the relevance of features associated with cluster affiliation. Results PAM clustering identified two clusters in the training dataset: Cluster 1 (n = 233) had a higher proportion of patients with higher sphericity and elongation, while Cluster 2 (n = 115) had a higher proportion of patients with higher maximum 3D diameter, surface area, axis lengths, and tumor volume (p < 0.001 for each). OS differed significantly between clusters: Cluster 1 showed a median OS of 23.8 compared to 11.4 months of Cluster 2 in the holdout test dataset (p = 0.002). Multivariate Cox regression showed improved performance with cluster affiliation over clinical data alone (C index 0.67 vs 0.59, p = 0.003). Cluster-based models outperformed the models with tumor volume alone (evidence ratio: 5.16–5.37). Conclusion Data-driven clustering reveals imaging phenotypes, highlighting the improved prognostic power of combining shape-radiomics with tumor volume, thereby outperforming predictions based on tumor volume alone in high-grade glioma survival outcomes. Clinical relevance statement Shape-radiomics and volume-based cluster analyses of preoperative MRI scans can reveal imaging phenotypes that improve the prediction of OS in patients with IDH-wild type gliomas, outperforming currently known models based on tumor size alone or clinical parameters. Key Points Shape radiomics and tumor volume clustering in IDH-wildtype gliomas are investigated for enhanced prognostic accuracy. Two distinct phenotypic clusters were identified with different median OSs. Integrating shape radiomics and volume-based clustering enhances OS prediction in IDH-wildtype glioma patients.

Enhancing the reliability of deep learning-based head and neck tumour segmentation using uncertainty estimation with multi-modal images

Objective. Deep learning shows promise in autosegmentation of head and neck cancer (HNC) primary tumours (GTV-T) and nodal metastases (GTV-N). However, errors such as including non-tumour regions or missing nodal metastases still occur. Conventional methods often make overconfident predictions, compromising reliability. Incorporating uncertainty estimation, which provides calibrated confidence intervals can address this issue. Our aim was to investigate the efficacy of various uncertainty estimation methods in improving segmentation reliability. We evaluated their confidence levels in voxel predictions and ability to reveal potential segmentation errors. Approach. We retrospectively collected data from 567 HNC patients with diverse cancer sites and multi-modality images (CT, PET, T1-, and T2-weighted MRI) along with their clinical GTV-T/N delineations. Using the nnUNet 3D segmentation pipeline, we compared seven uncertainty estimation methods, evaluating them based on segmentation accuracy (Dice similarity coefficient, DSC), confidence calibration (Expected Calibration Error, ECE), and their ability to reveal segmentation errors (Uncertainty-Error overlap using DSC, UE-DSC). Main results. Evaluated on the hold-out test dataset (n = 97), the median DSC scores for GTV-T and GTV-N segmentation across all uncertainty estimation methods had a narrow range, from 0.73 to 0.76 and 0.78 to 0.80, respectively. In contrast, the median ECE exhibited a wider range, from 0.30 to 0.12 for GTV-T and 0.25 to 0.09 for GTV-N. Similarly, the median UE-DSC also ranged broadly, from 0.21 to 0.38 for GTV-T and 0.22 to 0.36 for GTV-N. A probabilistic network—PhiSeg method consistently demonstrated the best performance in terms of ECE and UE-DSC. Significance. Our study highlights the importance of uncertainty estimation in enhancing the reliability of deep learning for autosegmentation of HNC GTV. The results show that while segmentation accuracy can be similar across methods, their reliability, measured by calibration error and uncertainty-error overlap, varies significantly. Used with visualisation maps, these methods may effectively pinpoint uncertainties and potential errors at the voxel level.

A Deep Transfer Learning Approach for Sleep Stage Classification and Sleep Apnea Detection Using Wrist-Worn Consumer Sleep Technologies.

Obstructive sleep apnea (OSA) is a common, underdiagnosed sleep-related breathing disorder with serious health implications Objective - We propose a deep transfer learning approach for sleep stage classification and sleep apnea (SA) detection using wrist-worn consumer sleep technologies (CST). Methods - Our model is based on a deep convolutional neural network (DNN) utilizing accelerometers and photo-plethysmography signals from nocturnal recordings. The DNN was trained and tested on internal datasets that include raw data from clinical and wrist-worn devices; external validation was performed on a hold-out test dataset containing raw data from a wrist-worn CST. Results - Training on clinical data improves performance significantly, and feature enrichment through a sleep stage stream gives only minor improvements. Raw data input outperforms feature-based input in CST datasets. The system generalizes well but performs slightly worse on wearable device data compared to clinical data. However, it excels in detecting events during REM sleep and is associated with arousal and oxygen desaturation. We found; cases that were significantly underestimated were characterized by fewer of such event associations. Conclusion - This study showcases the potential of using CSTs as alternate screening solution for undiagnosed cases of OSA. Significance - This work is significant for its development of a deep transfer learning approach using wrist-worn consumer sleep technologies, offering comprehensive validation for data utilization, and learning techniques, ultimately improving sleep apnea detection across diverse devices.

The Role of Artificial Intelligence in Predicting Optic Neuritis Subtypes From Ocular Fundus Photographs.

Optic neuritis (ON) is a complex clinical syndrome that has diverse etiologies and treatments based on its subtypes. Notably, ON associated with multiple sclerosis (MS ON) has a good prognosis for recovery irrespective of treatment, whereas ON associated with other conditions including neuromyelitis optica spectrum disorders or myelin oligodendrocyte glycoprotein antibody-associated disease is often associated with less favorable outcomes. Delay in treatment of these non-MS ON subtypes can lead to irreversible vision loss. It is important to distinguish MS ON from other ON subtypes early, to guide appropriate management. Yet, identifying ON and differentiating subtypes can be challenging as MRI and serological antibody test results are not always readily available in the acute setting. The purpose of this study is to develop a deep learning artificial intelligence (AI) algorithm to predict subtype based on fundus photographs, to aid the diagnostic evaluation of patients with suspected ON. This was a retrospective study of patients with ON seen at our institution between 2007 and 2022. Fundus photographs (1,599) were retrospectively collected from a total of 321 patients classified into 2 groups: MS ON (262 patients; 1,114 photographs) and non-MS ON (59 patients; 485 photographs). The dataset was divided into training and holdout test sets with an 80%/20% ratio, using stratified sampling to ensure equal representation of MS ON and non-MS ON patients in both sets. Model hyperparameters were tuned using 5-fold cross-validation on the training dataset. The overall performance and generalizability of the model was subsequently evaluated on the holdout test set. The receiver operating characteristic (ROC) curve for the developed model, evaluated on the holdout test dataset, yielded an area under the ROC curve of 0.83 (95% confidence interval [CI], 0.72-0.92). The model attained an accuracy of 76.2% (95% CI, 68.4-83.1), a sensitivity of 74.2% (95% CI, 55.9-87.4) and a specificity of 76.9% (95% CI, 67.6-85.0) in classifying images as non-MS-related ON. This study provides preliminary evidence supporting a role for AI in differentiating non-MS ON subtypes from MS ON. Future work will aim to increase the size of the dataset and explore the role of combining clinical and paraclinical measures to refine deep learning models over time.

Adaptive fine-tuning based transfer learning for the identification of MGMT promoter methylation status.

Background.Glioblastoma Multiforme (GBM) is an aggressive form of malignant brain tumor with a generally poor prognosis.O6-methylguanine-DNA methyltransferase (MGMT) promoter methylation has been shown to be a predictive bio-marker for resistance to treatment of GBM, but it is invasive and time-consuming to determine methylation status. There has been effort to predict the MGMT methylation status through analyzing MRI scans using machine learning, which only requires pre-operative scans that are already part of standard-of-care for GBM patients.Purpose.To improve the performance of conventional transfer learning in the identification of MGMT promoter methylation status, we developed a 3D SpotTune network with adaptive fine-tuning capability. Using the pretrained weights of MedicalNet with the SpotTune network, we compared its performance with a randomly initialized network for different combinations of MR modalities.Methods.Using a ResNet50 as the base network, three categories of networks are created: (1) A 3D SpotTune network to process volumetric MR images, (2) a network with randomly initialized weights, and (3) a network pre-trained on MedicalNet. These three networks are trained and evaluated using a public GBM dataset provided by the University of Pennsylvania. The MRI scans from 240 patients are used, with 11 different modalities corresponding to a set of perfusion, diffusion, and structural scans. The performance is evaluated using 5-fold cross validation with a hold-out testing dataset.Results.The SpotTune network showed better performance than the randomly initialized network. The best performing SpotTune model achieved an area under the Receiver Operating Characteristic curve (AUC), average precision of the precision-recall curve (AP), sensitivity, and specificity values of 0.6604, 0.6179, 0.6667, and 0.6061 respectively.Conclusions.SpotTune enables transfer learning to be adaptive to individual patients, resulting in improved performance in predicting MGMT promoter methylation status in GBM using equivalent MRI modalities as compared to a randomly initialized network.

A computational clinical decision-supporting system to suggest effective anti-epileptic drugs for pediatric epilepsy patients based on deep learning models using patient’s medical history

BackgroundEpilepsy, a chronic brain disorder characterized by abnormal brain activity that causes seizures and other symptoms, is typically treated using anti-epileptic drugs (AEDs) as the first-line therapy. However, due to the variations in their modes of action, identification of effective AEDs often relies on ad hoc trials, which is particularly challenging for pediatric patients. Thus, there is significant value in computational methods capable of assisting in the selection of AEDs, aiming to minimize unnecessary medication and improve treatment efficacy.ResultsIn this study, we collected 7,507 medical records from 1,000 pediatric epilepsy patients and developed a computational clinical decision-supporting system for AED selection. This system leverages three multi-channel convolutional neural network (CNN) models tailored to three specific AEDs (vigabatrin, prednisolone, and clobazam). Each CNN model predicts whether a respective AED is effective on a given patient or not. The CNN models showed AUROCs of 0.90, 0.80, and 0.92 in 10-fold cross-validation, respectively. Evaluation on a hold-out test dataset further revealed positive predictive values (PPVs) of 0.92, 0.97, and 0.91 for the three respective CNN models, representing that suggested AEDs by our models would be effective in controlling epilepsy with a high accuracy and thereby reducing unnecessary medications for pediatric patients.ConclusionOur CNN models in the system demonstrated high PPVs for the three AEDs, which signifies the potential of our approach to support the clinical decision-making by assisting doctors in recommending effective AEDs within the three AEDs for patients based on their medical history. This would result in a reduction in the number of unnecessary ad hoc attempts to find an effective AED for pediatric epilepsy patients.

Assessment on fast simulation of wind-driven pollutant dispersion around a street canyon with regime-switching Markov chain

High-density cities shorten commutes but also create proximity to air pollution sources. Sensors coupled with simulations can achieve real-time monitoring and risk assessment, and the recently emerging data-driven models provide opportunities that allow minimizing the computation burden of physical models. This study investigates the use of a time inhomogeneous Markov chain (CLF-MC) to fast simulation of wind-driven pollutant dispersion around a street canyon. The CLF-MC model is trained and tested using synthetic datasets obtained from large eddy simulations (LES). The LES model is validated using a benchmark wind tunnel test, which yields correlation coefficients above 0.95. The results show that the CLF-MC can faithfully reproduce the cumulative distribution of air pollution concentrations at the receptor site using the optimal model configuration identified in this study. For the hold-out test dataset, the release condition is a time-varying release strength, and the CLF-MC shows improved prediction results compared to the baseline models, i.e. the weighted average deviation is 15 % and 51 % smaller, respectively. In comparison to LES, the CLF-MC model is 4.8 × 106 times faster. These findings provide scientific references for fast and accurate wind-driven pollutant dispersion simulation using the probabilistic modelling approach.

Early Prediction of Regional Red Needle Cast Outbreaks Using Climatic Data Trends and Satellite-Derived Observations

Red needle cast (RNC), mainly caused by Phytophthora pluvialis, is a very damaging disease of the widely grown species radiata pine within New Zealand. Using a combination of satellite imagery and weather data, a novel methodology was developed to pre-visually predict the incidence of RNC on radiata pine within the Gisborne region of New Zealand over a five-year period from 2019 to 2023. Sentinel-2 satellite imagery was used to classify areas within the region as being disease-free or showing RNC expression from the difference in the red/green index (R/Gdiff) during a disease-free time of the year and the time of maximum disease expression in the upper canopy (early spring–September). Within these two classes, 1976 plots were extracted, and a classification model was used to predict disease incidence from mean monthly weather data for key variables during the 11 months prior to disease expression. The variables in the final random forest model included solar radiation, relative humidity, rainfall, and the maximum air temperature recorded during mid–late summer, which provided a pre-visual prediction of the disease 7–8 months before its peak expression. Using a hold-out test dataset, the final random forest model had an accuracy of 89% and an F1 score of 0.89. This approach can be used to mitigate the impact of RNC by focusing on early surveillance and treatment measures.

Prediction of pyrazinamide resistance in Mycobacterium tuberculosis using structure-based machine-learning approaches.

Pyrazinamide is one of four first-line antibiotics used to treat tuberculosis; however, antibiotic susceptibility testing for pyrazinamide is challenging. Resistance to pyrazinamide is primarily driven by genetic variation in pncA, encoding an enzyme that converts pyrazinamide into its active form. We curated a dataset of 664 non-redundant, missense amino acid mutations in PncA with associated high-confidence phenotypes from published studies and then trained three different machine-learning models to predict pyrazinamide resistance. All models had access to a range of protein structural-, chemical- and sequence-based features. The best model, a gradient-boosted decision tree, achieved a sensitivity of 80.2% and a specificity of 76.9% on the hold-out test dataset. The clinical performance of the models was then estimated by predicting the binary pyrazinamide resistance phenotype of 4027 samples harbouring 367 unique missense mutations in pncA derived from 24 231 clinical isolates. This work demonstrates how machine learning can enhance the sensitivity/specificity of pyrazinamide resistance prediction in genetics-based clinical microbiology workflows, highlights novel mutations for future biochemical investigation, and is a proof of concept for using this approach in other drugs.

Machine learning models to evaluate mortality in pediatric patients with pneumonia in the intensive care unit.

This study aimed to predict mortality in children with pneumonia who were admitted to the intensive care unit (ICU) to aid decision-making. Retrospective cohort study conducted at a single tertiary hospital. This study included children who were admitted to the pediatric ICU at the National Taiwan University Hospital between 2010 and 2019 due to pneumonia. Two prediction models were developed using tree-structured machine learning algorithms. The primary outcomes were ICU mortality and 24-h ICU mortality. A total of 33 features, including demographics, underlying diseases, vital signs, and laboratory data, were collected from the electronic health records. The machine learning models were constructed using the development data set, and performance matrices were computed using the holdout test data set. A total of 1231 ICU admissions of children with pneumonia were included in the final cohort. The area under the receiver operating characteristic curves (AUROCs) of the ICU mortality model and 24-h ICU mortality models was 0.80 (95% confidence interval [CI], 0.69-0.91) and 0.92 (95% CI, 0.86-0.92), respectively. Based on feature importance, the model developed in this study tended to predict increased mortality for the subsequent 24 h if a reduction in the blood pressure, peripheral capillary oxygen saturation (SpO2), or higher partial pressure of carbon dioxide (PCO2) were observed. This study demonstrated that the machine learning models for predicting ICU mortality and 24-h ICU mortality in children with pneumonia have the potential to support decision-making, especially in resource-limited settings.

Federated Learning for Decentralized Artificial Intelligence in Melanoma Diagnostics

The development of artificial intelligence (AI)-based melanoma classifiers typically calls for large, centralized datasets, requiring hospitals to give away their patient data, which raises serious privacy concerns. To address this concern, decentralized federated learning has been proposed, where classifier development is distributed across hospitals. To investigate whether a more privacy-preserving federated learning approach can achieve comparable diagnostic performance to a classical centralized (ie, single-model) and ensemble learning approach for AI-based melanoma diagnostics. This multicentric, single-arm diagnostic study developed a federated model for melanoma-nevus classification using histopathological whole-slide images prospectively acquired at 6 German university hospitals between April 2021 and February 2023 and benchmarked it using both a holdout and an external test dataset. Data analysis was performed from February to April 2023. All whole-slide images were retrospectively analyzed by an AI-based classifier without influencing routine clinical care. The area under the receiver operating characteristic curve (AUROC) served as the primary end point for evaluating the diagnostic performance. Secondary end points included balanced accuracy, sensitivity, and specificity. The study included 1025 whole-slide images of clinically melanoma-suspicious skin lesions from 923 patients, consisting of 388 histopathologically confirmed invasive melanomas and 637 nevi. The median (range) age at diagnosis was 58 (18-95) years for the training set, 57 (18-93) years for the holdout test dataset, and 61 (18-95) years for the external test dataset; the median (range) Breslow thickness was 0.70 (0.10-34.00) mm, 0.70 (0.20-14.40) mm, and 0.80 (0.30-20.00) mm, respectively. The federated approach (0.8579; 95% CI, 0.7693-0.9299) performed significantly worse than the classical centralized approach (0.9024; 95% CI, 0.8379-0.9565) in terms of AUROC on a holdout test dataset (pairwise Wilcoxon signed-rank, P < .001) but performed significantly better (0.9126; 95% CI, 0.8810-0.9412) than the classical centralized approach (0.9045; 95% CI, 0.8701-0.9331) on an external test dataset (pairwise Wilcoxon signed-rank, P < .001). Notably, the federated approach performed significantly worse than the ensemble approach on both the holdout (0.8867; 95% CI, 0.8103-0.9481) and external test dataset (0.9227; 95% CI, 0.8941-0.9479). The findings of this diagnostic study suggest that federated learning is a viable approach for the binary classification of invasive melanomas and nevi on a clinically representative distributed dataset. Federated learning can improve privacy protection in AI-based melanoma diagnostics while simultaneously promoting collaboration across institutions and countries. Moreover, it may have the potential to be extended to other image classification tasks in digital cancer histopathology and beyond.

Incorporating longitudinal history of risk factors into atherosclerotic cardiovascular disease risk prediction using deep learning

It is increasingly clear that longitudinal risk factor levels and trajectories are related to risk for atherosclerotic cardiovascular disease (ASCVD) above and beyond single measures. Currently used in clinical care, the Pooled Cohort Equations (PCE) are based on regression methods that predict ASCVD risk based on cross-sectional risk factor levels. Deep learning (DL) models have been developed to incorporate longitudinal data for risk prediction but its benefit for ASCVD risk prediction relative to the traditional Pooled Cohort Equations (PCE) remain unknown. Our study included 15,565 participants from four cardiovascular disease cohorts free of baseline ASCVD who were followed for adjudicated ASCVD. Ten-year ASCVD risk was calculated in the training set using our benchmark, the PCE, and a longitudinal DL model, Dynamic-DeepHit. Predictors included those incorporated in the PCE: sex, race, age, total cholesterol, high density lipid cholesterol, systolic and diastolic blood pressure, diabetes, hypertension treatment and smoking. The discrimination and calibration performance of the two models were evaluated in an overall hold-out testing dataset. Of the 15,565 participants in our dataset, 2170 (13.9%) developed ASCVD. The performance of the longitudinal DL model that incorporated 8 years of longitudinal risk factor data improved upon that of the PCE [AUROC: 0.815 (CI 0.782–0.844) vs 0.792 (CI 0.760–0.825)] and the net reclassification index was 0.385. The brier score for the DL model was 0.0514 compared with 0.0542 in the PCE. Incorporating longitudinal risk factors in ASCVD risk prediction using DL can improve model discrimination and calibration.

Deep learning-based automatic segmentation of the mandibular canal on panoramic radiographs: A multi-device study.

The objective of this study was to propose a deep-learning model for the detection of the mandibular canal on dental panoramic radiographs. A total of 2,100 panoramic radiographs (PANs) were collected from 3 different machines: RAYSCAN Alpha (n=700, PAN A), OP-100 (n=700, PAN B), and CS8100 (n=700, PAN C). Initially, an oral and maxillofacial radiologist coarsely annotated the mandibular canals. For deep learning analysis, convolutional neural networks (CNNs) utilizing U-Net architecture were employed for automated canal segmentation. Seven independent networks were trained using training sets representing all possible combinations of the 3 groups. These networks were then assessed using a hold-out test dataset. Among the 7 networks evaluated, the network trained with all 3 available groups achieved an average precision of 90.6%, a recall of 87.4%, and a Dice similarity coefficient (DSC) of 88.9%. The 3 networks trained using each of the 3 possible 2-group combinations also demonstrated reliable performance for mandibular canal segmentation, as follows: 1) PAN A and B exhibited a mean DSC of 87.9%, 2) PAN A and C displayed a mean DSC of 87.8%, and 3) PAN B and C demonstrated a mean DSC of 88.4%. This multi-device study indicated that the examined CNN-based deep learning approach can achieve excellent canal segmentation performance, with a DSC exceeding 88%. Furthermore, the study highlighted the importance of considering the characteristics of panoramic radiographs when developing a robust deep-learning network, rather than depending solely on the size of the dataset.

Explainable machine learning for predicting conversion to neurological disease: Results from 52,939 medical records.

This study assesses the application of interpretable machine learning modeling using electronic medical record data for the prediction of conversion to neurological disease. A retrospective dataset of Cleveland Clinic patients diagnosed with Alzheimer's disease, amyotrophic lateral sclerosis, multiple sclerosis, or Parkinson's disease, and matched controls based on age, sex, race, and ethnicity was compiled. Individualized risk prediction models were created using eXtreme Gradient Boosting for each neurological disease at four timepoints in patient history. The prediction models were assessed for transparency and fairness. At timepoints 0-months, 12-months, 24-months, and 60-months prior to diagnosis, Alzheimer's disease models achieved the area under the receiver operating characteristic curve on a holdout test dataset of 0.794, 0.742, 0.709, and 0.645; amyotrophic lateral sclerosis of 0.883, 0.710, 0.658, and 0.620; multiple sclerosis of 0.922, 0.877, 0.849, and 0.781; and Parkinson's disease of 0.809, 0.738, 0.700, and 0.651, respectively. The results demonstrate that electronic medical records contain latent information that can be used for risk stratification for neurological disorders. In particular, patient-reported outcomes, sleep assessments, falls data, additional disease diagnoses, and longitudinal changes in patient health, such as weight change, are important predictors.

A Machine Learning Algorithm Predicting Acute Kidney Injury in Intensive Care Unit Patients (NAVOY Acute Kidney Injury): Proof-of-Concept Study.

Acute kidney injury (AKI) represents a significant global health challenge, leading to increased patient distress and financial health care burdens. The development of AKI in intensive care unit (ICU) settings is linked to prolonged ICU stays, a heightened risk of long-term renal dysfunction, and elevated short- and long-term mortality rates. The current diagnostic approach for AKI is based on late indicators, such as elevated serum creatinine and decreased urine output, which can only detect AKI after renal injury has transpired. There are no treatments to reverse or restore renal function once AKI has developed, other than supportive care. Early prediction of AKI enables proactive management and may improve patient outcomes. The primary aim was to develop a machine learning algorithm, NAVOY Acute Kidney Injury, capable of predicting the onset of AKI in ICU patients using data routinely collected in ICU electronic health records. The ultimate goal was to create a clinical decision support tool that empowers ICU clinicians to proactively manage AKI and, consequently, enhance patient outcomes. We developed the NAVOY Acute Kidney Injury algorithm using a hybrid ensemble model, which combines the strengths of both a Random Forest (Leo Breiman and Adele Cutler) and an XGBoost model (Tianqi Chen). To ensure the accuracy of predictions, the algorithm used 22 clinical variables for hourly predictions of AKI as defined by the Kidney Disease: Improving Global Outcomes guidelines. Data for algorithm development were sourced from the Massachusetts Institute of Technology Lab for Computational Physiology Medical Information Mart for Intensive Care IV clinical database, focusing on ICU patients aged 18 years or older. The developed algorithm, NAVOY Acute Kidney Injury, uses 4 hours of input and can, with high accuracy, predict patients with a high risk of developing AKI 12 hours before onset. The prediction performance compares well with previously published prediction algorithms designed to predict AKI onset in accordance with Kidney Disease: Improving Global Outcomes diagnosis criteria, with an impressive area under the receiver operating characteristics curve (AUROC) of 0.91 and an area under the precision-recall curve (AUPRC) of 0.75. The algorithm's predictive performance was externally validated on an independent hold-out test data set, confirming its ability to predict AKI with exceptional accuracy. NAVOY Acute Kidney Injury is an important development in the field of critical care medicine. It offers the ability to predict the onset of AKI with high accuracy using only 4 hours of data routinely collected in ICU electronic health records. This early detection capability has the potential to strengthen patient monitoring and management, ultimately leading to improved patient outcomes. Furthermore, NAVOY Acute Kidney Injury has been granted Conformite Europeenne (CE)-marking, marking a significant milestone as the first CE-marked AKI prediction algorithm for commercial use in European ICUs.

Electrocardiographic deep learning for predicting post-procedural mortality: a model development and validation study

Preoperative risk assessments used in clinical practice are insufficient in their ability to identify risk for postoperative mortality. Deep-learning analysis of electrocardiography can identify hidden risk markers that can help to prognosticate postoperative mortality. We aimed to develop a prognostic model that accurately predicts postoperative mortality in patients undergoing medical procedures and who had received preoperative electrocardiographic diagnostic testing. In a derivation cohort of preoperative patients with available electrocardiograms (ECGs) from Cedars-Sinai Medical Center (Los Angeles, CA, USA) between Jan 1, 2015 and Dec 31, 2019, a deep-learning algorithm was developed to leverage waveform signals to discriminate postoperative mortality. We randomly split patients (8:1:1) into subsets for training, internal validation, and final algorithm test analyses. Model performance was assessed using area under the receiver operating characteristic curve (AUC) values in the hold-out test dataset and in two external hospital cohorts and compared with the established Revised Cardiac Risk Index (RCRI) score. The primary outcome was post-procedural mortality across three health-care systems. 45 969 patients had a complete ECG waveform image available for at least one 12-lead ECG performed within the 30 days before the procedure date (59 975 inpatient procedures and 112 794 ECGs): 36 839 patients in the training dataset, 4549 in the internal validation dataset, and 4581 in the internal test dataset. In the held-out internal test cohort, the algorithm discriminates mortality with an AUC value of 0·83 (95% CI 0·79-0·87), surpassing the discrimination of the RCRI score with an AUC of 0·67 (0·61-0·72). The algorithm similarly discriminated risk for mortality in two independent US health-care systems, with AUCs of 0·79 (0·75-0·83) and 0·75 (0·74-0·76), respectively. Patients determined to be high risk by the deep-learning model had an unadjusted odds ratio (OR) of 8·83 (5·57-13·20) for postoperative mortality compared with an unadjusted OR of 2·08 (0·77-3·50) for postoperative mortality for RCRI scores of more than 2. The deep-learning algorithm performed similarly for patients undergoing cardiac surgery (AUC 0·85 [0·77-0·92]), non-cardiac surgery (AUC 0·83 [0·79-0·88]), and catheterisation or endoscopy suite procedures (AUC 0·76 [0·72-0·81]). A deep-learning algorithm interpreting preoperative ECGs can improve discrimination of postoperative mortality. The deep-learning algorithm worked equally well for risk stratification of cardiac surgeries, non-cardiac surgeries, and catheterisation laboratory procedures, and was validated in three independent health-care systems. This algorithm can provide additional information to clinicians making the decision to perform medical procedures and stratify the risk of future complications. National Heart, Lung, and Blood Institute.

Abstract 18718: A Deep Learning Model for Classifying Left Ventricular Enlargement Using Transthoracic and Point-of-Care Echocardiograms

Background: Enlargement or dilation of the left ventricle (LV) is associated with increased cardiovascular morbidity and mortality, and is an established precursor of ventricular dysfunction and heart failure in almost all major cardiovascular conditions. Although there are existing artificial intelligence frameworks that make use of deep learning to detect LV enlargement, many entail LV segmentation, which can be error-prone especially for point-of-care ultrasound (POCUS) images. Goal: To develop a fully end-to-end deep learning framework that takes as input either transthoracic echocardiographic (TTE) or POCUS 2D image clips, and classifies directly (i.e., without first segmenting the LV) whether a patient has an enlarged LV. Methods: Using a sample of retrospective TTE data enriched for LV enlargement (out of 13,868 patients total, 3,351 had an enlarged LV defined using sex-specific guideline-directed criteria), we trained a classifier that takes as input apical 2-chamber and parasternal long axis videos, and provides as output a binary label denoting whether a patient has an enlarged LV. Then, we evaluated the trained model using a holdout testing dataset of retrospective TTE data (2,087 patients) as well as data from a prospective cohort of 428 patients from whom TTE and POCUS were collected simultaneously. Results: For the retrospective TTE testing dataset, the AUC was 0.851 (95% confidence interval (CI) of 0.830-871). For the prospective cohort, the model performed comparably for both TTE and POCUS video clips (TTE: AUC of 0.865, 95% CI of 0.823-0.906; POCUS: AUC of 0.840, 95% CI of 0.797-0.883). Conclusions: The current deep learning model performed comparably well for both TTE and POCUS echocardiograms. We regard these results as a proof-of-concept of the potential utility of this model in helping point-of-care clinicians make decisions regarding whether patients should receive further evaluation for LV enlargement and associated conditions.

Trustworthy Data and AI Environments for Clinical Prediction: Application to Crisis-Risk in People with Depression.

Depression is a common mental health condition that often occurs in association with other chronic illnesses, and varies considerably in severity. Electronic Health Records (EHRs) contain rich information about a patient's medical history and can be used to train, test and maintain predictive models to support and improve patient care. This work evaluated the feasibility of implementing an environment for predicting mental health crisis among people living with depression based on both structured and unstructured EHRs. A large EHR from a mental health provider, Mersey Care, was pseudonymised and ingested into the Natural Language Processing (NLP) platform CogStack, allowing text content in binary clinical notes to be extracted. All unstructured clinical notes and summaries were semantically annotated by MedCAT and BioYODIE NLP services. Cases of crisis in patients with depression were then identified. Random forest models, gradient boosting trees, and Long Short-Term Memory (LSTM) networks, with varying feature arrangement, were trained to predict the occurrence of crisis. The results showed that all the prediction models can use a combination of structured and unstructured EHR information to predict crisis in patients with depression with good and useful accuracy. The LSTM network that was trained on a modified dataset with only 1000 most-important features from the random forest model with temporality showed the best performance with a mean AUC of 0.901 and a standard deviation of 0.006 using a training dataset and a mean AUC of 0.810 and 0.01 using a hold-out test dataset. Comparing the results from the technical evaluation with the views of psychiatrists shows that there are now opportunities to refine and integrate such prediction models into pragmatic point-of-care clinical decision support tools for supporting mental healthcare delivery.

Data-driven wake model parameter estimation to analyze effects of wake superposition

Low-fidelity wake models are used for wind farm design and control optimization. To generalize to a wind farm model, individually modeled wakes are commonly superimposed using approximate superposition models. Wake models parameterize atmospheric and wake turbulence, introducing unknown model parameters that historically are tuned with idealized simulation or experimental data and neglect uncertainty. We calibrate and estimate the uncertainty of the parameters in a Gaussian wake model using Markov chain Monte Carlo (MCMC) for various wake superposition methods. Posterior distributions of the uncertain parameters are generated using power production data from large eddy simulations and a utility-scale wake steering field experiment. The posteriors for the wake expansion coefficient are sensitive to the choice of superposition method, with relative differences in the means and standard deviations on the order of 100%. This sensitivity illustrates the role of superposition methods in wake modeling error. We compare these data-driven parameter estimates to estimates derived from a standard turbulence-intensity based model as a baseline. To assess predictive accuracy, we calibrate the data-driven parameter estimates with a training dataset for yaw-aligned operation. Using a Monte Carlo approach, we then generate predicted distributions of turbine power production and evaluate against a hold-out test dataset for yaw-misaligned operation. For the cases tested, the MCMC-calibrated parameters reduce the total error of the power predictions by roughly 50% compared to the deterministic empirical model predictions. An additional benefit of the data-driven parameter estimation is the quantification of uncertainty, which enables physically quantified confidence intervals of wake model predictions.