Average Area Under The ROC Curve Research Articles

Impact of 18F-FDG PET Intensity Normalization on Radiomic Features of Oropharyngeal Squamous Cell Carcinomas and Machine Learning-Generated Biomarkers.

We aimed to investigate the effects of 18F-FDG PET voxel intensity normalization on radiomic features of oropharyngeal squamous cell carcinoma (OPSCC) and machine learning-generated radiomic biomarkers. Methods: We extracted 1,037 18F-FDG PET radiomic features quantifying the shape, intensity, and texture of 430 OPSCC primary tumors. The reproducibility of individual features across 3 intensity-normalized images (body-weight SUV, reference tissue activity ratio to lentiform nucleus of brain and cerebellum) and the raw PET data was assessed using an intraclass correlation coefficient (ICC). We investigated the effects of intensity normalization on the features' utility in predicting the human papillomavirus (HPV) status of OPSCCs in univariate logistic regression, receiver-operating-characteristic analysis, and extreme-gradient-boosting (XGBoost) machine-learning classifiers. Results: Of 1,037 features, a high (ICC ≥ 0.90), medium (0.90 > ICC ≥ 0.75), and low (ICC < 0.75) degree of reproducibility across normalization methods was attained in 356 (34.3%), 608 (58.6%), and 73 (7%) features, respectively. In univariate analysis, features from the PET normalized to the lentiform nucleus had the strongest association with HPV status, with 865 of 1,037 (83.4%) significant features after multiple testing corrections and a median area under the receiver-operating-characteristic curve (AUC) of 0.65 (interquartile range, 0.62-0.68). Similar tendencies were observed in XGBoost models, with the lentiform nucleus-normalized model achieving the numerically highest average AUC of 0.72 (SD, 0.07) in the cross validation within the training cohort. The model generalized well to the validation cohorts, attaining an AUC of 0.73 (95% CI, 0.60-0.85) in independent validation and 0.76 (95% CI, 0.58-0.95) in external validation. The AUCs of the XGBoost models were not significantly different. Conclusion: Only one third of the features demonstrated a high degree of reproducibility across intensity-normalization techniques, making uniform normalization a prerequisite for interindividual comparability of radiomic markers. The choice of normalization technique may affect the radiomic features' predictive value with respect to HPV. Our results show trends that normalization to the lentiform nucleus may improve model performance, although more evidence is needed to draw a firm conclusion.

One-shot screening: Utilization of a two-dimensional convolutional neural network for automatic detection of left ventricular hypertrophy using electrocardiograms

Background and ObjectiveLeft ventricular hypertrophy (LVH) can impair ejection function and elevate the risk of heart failure. Therefore, early detection through screening is crucial. This study aimed to propose a novel method to enhance LVH detection using 12-lead electrocardiogram (ECG) waveforms with a two-dimensional (2D) convolutional neural network (CNN). MethodsUtilizing 42,127 pairs of ECG-transthoracic echocardiogram data, we pre-processed raw data into single-shot images derived from each ECG lead and conducted lead selection to optimize LVH diagnosis. Our proposed one-shot screening method, implemented during pre-processing, enables the superimposition of waveform source data of any length onto a single-frame image, thereby addressing the limitations of the one-dimensional (1D) approach. We developed a deep learning model with a 2D-CNN structure and machine learning models for LVH detection. To assess our method, we also compared our results with conventional ECG criteria and those of a prior study that used a 1D-CNN approach, utilizing the same dataset from the University of Tokyo Hospital for LVH diagnosis. ResultsFor LVH detection, the average area under the receiver operating characteristic curve (AUROC) was 0.916 for the 2D-CNN model, which was significantly higher than that obtained using logistic regression and random forest methods, as well as the two conventional ECG criteria (AUROC of 0.766, 0.790, 0.599, and 0.622, respectively). Incorporating additional metadata, such as ECG measurement data, further improved the average AUROC to 0.921. The model's performance remained stable across two different annotation criteria and demonstrated significant superiority over the performance of the 1D-CNN model used in a previous study (AUROC of 0.807). ConclusionsThis study introduces a robust and computationally efficient method that outperforms 1D-CNN models utilized in previous studies for LVH detection. Our method can transform waveforms of any length into fixed-size images and leverage the selected lead of the ECG, ensuring adaptability in environments with limited computational resources. The proposed method holds promise for integration into clinical practice as a tool for early diagnosis, potentially enhancing patient outcomes by facilitating earlier treatment and management.

Improving the precision of shock resuscitation by predicting fluid responsiveness with machine learning and arterial blood pressure waveform data

Fluid bolus therapy (FBT) is fundamental to the management of circulatory shock in critical care but balancing the benefits and toxicities of FBT has proven challenging in individual patients. Improved predictors of the hemodynamic response to a fluid bolus, commonly referred to as a fluid challenge, are needed to limit non-beneficial fluid administration and to enable automated clinical decision support and patient-specific precision critical care management. In this study we retrospectively analyzed data from 394 fluid boluses from 58 pigs subjected to either hemorrhagic or distributive shock. All animals had continuous blood pressure and cardiac output monitored throughout the study. Using this data, we developed a machine learning (ML) model to predict the hemodynamic response to a fluid challenge using only arterial blood pressure waveform data as the input. A Random Forest binary classifier referred to as the ML fluid responsiveness algorithm (MLFRA) was trained to detect fluid responsiveness (FR), defined as a ≥ 15% change in cardiac stroke volume after a fluid challenge. We then compared its performance to pulse pressure variation, a commonly used metric of FR. Model performance was assessed using the area under the receiver operating characteristic curve (AUROC), confusion matrix metrics, and calibration curves plotting predicted probabilities against observed outcomes. Across multiple train/test splits and feature selection methods designed to assess performance in the setting of small sample size conditions typical of large animal experiments, the MLFRA achieved an average AUROC, recall (sensitivity), specificity, and precision of 0.82, 0.86, 0.62. and 0.76, respectively. In the same datasets, pulse pressure variation had an AUROC, recall, specificity, and precision of 0.73, 0.91, 0.49, and 0.71, respectively. The MLFRA was generally well-calibrated across its range of predicted probabilities and appeared to perform equally well across physiologic conditions. These results suggest that ML, using only inputs from arterial blood pressure monitoring, may substantially improve the accuracy of predicting FR compared to the use of pulse pressure variation. If generalizable, these methods may enable more effective, automated precision management of critically ill patients with circulatory shock.

A Deep Learning Model for Detecting Rhegmatogenous Retinal Detachment Using Ophthalmologic Ultrasound Images

Introduction: Rhegmatogenous retinal detachment (RRD) is one of the most common fundus diseases. Many rural areas of China have few ophthalmologists, and ophthalmologic ultrasound examination is of great significance for remote diagnosis of RRD. Therefore, this study aimed to develop and evaluate a deep learning (DL) model, to be used for automated RRD diagnosis based on ophthalmologic ultrasound images, in order to support timely diagnosis of RRD in rural and remote areas. Methods: A total of 6,000 ophthalmologic ultrasound images from 1,645 participants were used to train and verify the DL model. A total of 5,000 images were used for training and validating DL models, and an independent testing set of 1,000 images was used to test the performance of eight DL models trained using four different DL model architectures (fully connected neural network, LeNet5, AlexNet, and VGG16) and two preprocessing techniques (original, original image augmented). Receiver operating characteristic (ROC) curves were used to analyze their performance. Heatmaps were generated to visualize the process of the best DL model in the identification of RRD. Finally, five ophthalmologists were invited to diagnose RRD independently on the same test set of 1,000 images for performance comparison with the best DL model. Results: The best DL model for identifying RRD achieved an area under the ROC curve (AUC) of 0.998 with a sensitivity and specificity of 99.2% and 99.8%, respectively. The best preprocessing method in each model architecture was the application of original image augmentation (average AUC = 0.982). The best model architecture in each preprocessing method was VGG16 (average AUC = 0.998). Conclusion: The best DL model determined in this study has higher accuracy, sensitivity, and specificity than the ophthalmologists’ diagnosis in identifying RRD based on ophthalmologic ultrasound images. This model may provide support for timely diagnosis in locations without access to ophthalmologic care.

Prediction of chronic kidney disease progression using recurrent neural network and electronic health records

Chronic kidney disease (CKD) is a progressive loss in kidney function. Early detection of patients who will progress to late-stage CKD is of paramount importance for patient care. To address this, we develop a pipeline to process longitudinal electronic heath records (EHRs) and construct recurrent neural network (RNN) models to predict CKD progression from stages II/III to stages IV/V. The RNN model generates predictions based on time-series records of patients, including repeated lab tests and other clinical variables. Our investigation reveals that using a single variable, the recorded estimated glomerular filtration rate (eGFR) over time, the RNN model achieves an average area under the receiver operating characteristic curve (AUROC) of 0.957 for predicting future CKD progression. When additional clinical variables, such as demographics, vital information, lab test results, and health behaviors, are incorporated, the average AUROC increases to 0.967. In both scenarios, the standard deviation of the AUROC across cross-validation trials is less than 0.01, indicating a stable and high prediction accuracy. Our analysis results demonstrate the proposed RNN model outperforms existing standard approaches, including static and dynamic Cox proportional hazards models, random forest, and LightGBM. The utilization of the RNN model and the time-series data of previous eGFR measurements underscores its potential as a straightforward and effective tool for assessing the clinical risk of CKD patients concerning their disease progression.

MRI-based automated multitask deep learning system to evaluate supraspinatus tendon injuries.

To establish an automated, multitask, MRI-based deep learning system for the detailed evaluation of supraspinatus tendon (SST) injuries. According to arthroscopy findings, 3087 patients were divided into normal, degenerative, and tear groups (groups 0-2). Group 2 was further divided into bursal-side, articular-side, intratendinous, and full-thickness tear groups (groups 2.1-2.4), and external validation was performed with 573 patients. Visual geometry group network 16 (VGG16) was used for preliminary image screening. Then, the rotator cuff multitask learning (RC-MTL) model performed multitask classification (classifiers 1-4). A multistage decision model produced the final output. Model performance was evaluated by receiver operating characteristic (ROC) curve analysis and calculation of related parameters. McNemar's test was used to compare the differences in the diagnostic effects between radiologists and the model. The intraclass correlation coefficient (ICC) was used to assess the radiologists' reliability. p < 0.05 indicated statistical significance. In the in-group dataset, the area under the ROC curve (AUC) of VGG16 was 0.92, and the average AUCs of RC-MTL classifiers 1-4 were 0.99, 0.98, 0.97, and 0.97, respectively. The average AUC of the automated multitask deep learning system for groups 0-2.4 was 0.98 and 0.97 in the in-group and out-group datasets, respectively. The ICCs of the radiologists were 0.97-0.99. The automated multitask deep learning system outperformed the radiologists in classifying groups 0-2.4 in both the in-group and out-group datasets (p < 0.001). The MRI-based automated multitask deep learning system performed well in diagnosing SST injuries and is comparable to experienced radiologists. Our study established an automated multitask deep learning system to evaluate supraspinatus tendon (SST) injuries and further determine the location of SST tears. The model can potentially improve radiologists' diagnostic efficiency, reduce diagnostic variability, and accurately assess SST injuries. • A detailed classification of supraspinatus tendon tears can help clinical decision-making. • Deep learning enables the detailed classification of supraspinatus tendon injuries. • The proposed automated multitask deep learning system is comparable to radiologists.

Abstract 12374: Machine Learning-Based Evaluation of Elevated Left Ventricular End-Diastolic Pressure

Background: Left ventricular end-diastolic pressure (LVEDP) is clinically relevant due to its association with diastolic dysfunction and other outcomes. The gold-standard method of measuring LVEDP is invasive intra-cardiac catheterization. Echocardiography is routinely used for non-invasive estimation of left ventricular (LV) filling pressures; however, correlation with invasive LVEDP is variable. Goal: We sought to use machine learning (ML) algorithms to predict elevated LVEDP (&gt; 20 mmHg) using clinical, echocardiographic, and biochemical parameters. Methods: We used a cohort of 460 consecutive patients without atrial fibrillation or significant mitral valve disease who underwent transthoracic echocardiography within 24 hours of elective heart catheterization. We included patients’ clinical (e.g. age), echocardiographic (e.g. left ventricular ejection fraction), and biochemical (e.g. N-terminal brain natriuretic peptide) data. We fit logistic regression, random forest, gradient boosting, and support vector machine algorithms in a 20-iteration train-validate-test workflow and measured performance using average area under the receiver operating characteristic curve (AUROC). We also performed multi-class classification of the patients’ cardiac conditions and predicted elevated tau (&gt; 45), the gold-standard parameter for LV diastolic dysfunction. For each outcome, logistic regression weights were used to identify clinically relevant variables. Results: ML algorithms predicted LVEDP &gt; 20 mmHg with good performance (AUROC = 0.779). ML models showed excellent performance predicting elevated tau (AUROC = 0.830) and classifying a variety of cardiac conditions (AUROC = 0.837 - 0.973). We identified several clinical variables (e.g. BMI, heart rate) relevant to LVEDP prediction. Conclusion: Our study shows ML approaches can robustly predict elevated LVEDP and tau. ML may assist in the clinical interpretation of echocardiographic data.

Classification of Fracture Risk in Fallers Using Dual-Energy X-Ray Absorptiometry (DXA) Images and Deep Learning-Based Feature Extraction.

Dual-energy X-ray absorptiometry (DXA) scans are one of the most frequently used imaging techniques for calculating bone mineral density, yet calculating fracture risk using DXA image features is rarely performed. The objective of this study was to combine deep neural networks, together with DXA images and patient clinical information, to evaluate fracture risk in a cohort of adults with at least one known fall and age-matched healthy controls. DXA images of the entire body as, well as isolated images of the hip, forearm, and spine (1488 total), were obtained from 478 fallers and 48 non-faller controls. A modeling pipeline was developed for fracture risk prediction using the DXA images and clinical data. First, self-supervised pretraining of feature extractors was performed using a small vision transformer (ViT-S) and a convolutional neural network model (VGG-16 and Resnet-50). After pretraining, the feature extractors were then paired with a multilayer perceptron model, which was used for fracture risk classification. Classification was achieved with an average area under the receiver-operating characteristic curve (AUROC) score of 74.3%. This study demonstrates ViT-S as a promising neural network technique for fracture risk classification using DXA scans. The findings have future application as a fracture risk screening tool for older adults at risk of falls. © 2023 The Authors. JBMR Plus published by Wiley Periodicals LLC on behalf of American Society for Bone and Mineral Research.

Automatic intracranial abnormality detection and localization in head CT scans by learning from free-text reports

Deep learning has yielded promising results for medical image diagnosis but relies heavily on manual image annotations, which are expensive to acquire. We present Cross-DL, a cross-modality learning framework for intracranial abnormality detection and localization in head computed tomography (CT) scans by learning from free-text imaging reports. Cross-DL has a discretizer that automatically extracts discrete labels of abnormality types and locations from reports, which are utilized to train an image analyzer by a dynamic multi-instance learning approach. Benefiting from the low annotation cost and a consequent large-scale training set of 28,472 CT scans, Cross-DL achieves accurate performance, with an average area under the receiver operating characteristic curve (AUROC) of 0.956 (95% confidence interval: 0.952-0.959) in detecting 4 abnormality types in 17 regions while accurately localizing abnormalities at the voxel level. An intracranial hemorrhage classification experiment on the external dataset CQ500 achieves an AUROC of 0.928 (0.905-0.951). The model can also help review prioritization.

Prediction of the Presence of Ventricular Fibrillation From a Brugada Electrocardiogram Using Artificial Intelligence.

Brugada syndrome is a potential cause of sudden cardiac death (SCD) and is characterized by a distinct ECG, but not all patients with A Brugada ECG develop SCD. In this study we sought to examine if an artificial intelligence (AI) model can predict a previous or future ventricular fibrillation (VF) episode from a Brugada ECG.Methods and Results: We developed an AI-enabled algorithm using a convolutional neural network. From 157 patients with suspected Brugada syndrome, 2,053 ECGs were obtained, and the dataset was divided into 5 datasets for cross-validation. In the ECG-based evaluation, the precision, recall, and F1score were 0.79±0.09, 0.73±0.09, and 0.75±0.09, respectively. The average area under the receiver-operating characteristic curve (AUROC) was 0.81±0.09. On per-patient evaluation, the AUROC was 0.80±0.07. This model predicted the presence of VF with a precision of 0.93±0.02, recall of 0.77±0.14, and F1score of 0.81±0.11. The negative predictive value was 0.94±0.11 while its positive predictive value was 0.44±0.29. This proof-of-concept study showed that an AI-enabled algorithm can predict the presence of VF with a substantial performance. It implies that the AI model may detect a subtle ECG change that is undetectable by humans.

Practical intelligent diagnostic algorithm for wearable 12-lead ECG via self-supervised learning on large-scale dataset

Cardiovascular disease is a major global public health problem, and intelligent diagnostic approaches play an increasingly important role in the analysis of electrocardiograms (ECGs). Convenient wearable ECG devices enable the detection of transient arrhythmias and improve patient health by making it possible to seek intervention during continuous monitoring. We collected 658,486 wearable 12-lead ECGs, among which 164,538 were annotated, and the remaining 493,948 were without diagnostic. We present four data augmentation operations and a self-supervised learning classification framework that can recognize 60 ECG diagnostic terms. Our model achieves an average area under the receiver-operating characteristic curve (AUROC) and average F1 score on the offline test of 0.975 and 0.575. The average sensitivity, specificity and F1-score during the 2-month online test are 0.736, 0.954 and 0.468, respectively. This approach offers real-time intelligent diagnosis, and detects abnormal segments in long-term ECG monitoring in the clinical setting for further diagnosis by cardiologists.

Representation of time-varying and time-invariant EMR data and its application in modeling outcome prediction for heart failure patients

ObjectiveTo represent a patient record with both time-invariant and time-varying features as a single vector using an end-to-end deep learning model, and further to predict the kidney failure (KF) status and mortality of heart failure (HF) patients. Materials and methodsThe time-invariant EMR data included demographic information and comorbidities, and the time-varying EMR data were lab tests. We used a Transformer encoder module to represent the time-invariant data, and refined a long short-term memory (LSTM) with a Transformer encoder attached to the top to represent the time-varying data, taking the original measured values and their corresponding embedding vectors, masking vectors, and two types of time intervals as inputs. The proposed representations of patients with time-invariant and time-varying data were used to predict KF status (949 out of 5268 HF patients diagnosed with KF) and mortality (463 in-hospital deaths) for HF patients. Comparative experiments were conducted between the proposed model and some representative machine learning models. Ablation experiments were also performed around the time-varying data representation, including replacing the refined LSTM with the standard LSTM, GRU-D and T-LSTM, respectively, and removing the Transformer encoder and the time-varying data representation module, respectively. The visualization of the attention weights of the time-invariant and time-varying features was used to clinically interpret the predictive performance. We used the area under the receiver operating characteristic curve (AUROC), the area under the precision-recall curve (AUPRC), and the F1-score to evaluate the predictive performance of the models. ResultsThe proposed model achieved superior performance, with average AUROCs, AUPRCs and F1-scores of 0.960, 0.610 and 0.759 for KF prediction and 0.937, 0.353 and 0.537 for mortality prediction, respectively. Predictive performance improved with the addition of time-varying data from longer time periods. The proposed model outperformed the comparison and ablation references in both prediction tasks. ConclusionsBoth time-invariant and time-varying EMR data of patients could be efficiently represented by the proposed unified deep learning model, which shows higher performance in clinical prediction tasks. The way to use time-varying data in the current study is hopeful to be used in other kinds of time-varying data and other clinical tasks.

SynthEye: Investigating the Impact of Synthetic Data on Artificial Intelligence-assisted Gene Diagnosis of Inherited Retinal Disease.

Rare disease diagnosis is challenging in medical image-based artificial intelligence due to a natural class imbalance in datasets, leading to biased prediction models. Inherited retinal diseases (IRDs) are a research domain that particularly faces this issue. This study investigates the applicability of synthetic data in improving artificial intelligence-enabled diagnosis of IRDs using generative adversarial networks (GANs). Diagnostic study of gene-labeled fundus autofluorescence (FAF) IRD images using deep learning. Moorfields Eye Hospital (MEH) dataset of 15692 FAF images obtained from 1800 patients with confirmed genetic diagnosis of 1 of 36 IRD genes. A StyleGAN2 model is trained on the IRD dataset to generate 512× 512 resolution images. Convolutional neural networks are trained for classification using different synthetically augmented datasets, including real IRD images plus 1800 and 3600 synthetic images, and a fully rebalanced dataset. We also perform an experiment with only synthetic data. All models are compared against a baseline convolutional neural network trained only on real data. We evaluated synthetic data quality using a Visual Turing Test conducted with 4 ophthalmologists from MEH. Synthetic and real images were compared using feature space visualization, similarity analysis to detect memorized images, and Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE) score for no-reference-based quality evaluation. Convolutional neural network diagnostic performance was determined on a held-out test set using the area under the receiver operating characteristic curve (AUROC) and Cohen's Kappa (κ). An average true recognition rate of 63% and fake recognition rate of 47% was obtained from the Visual Turing Test. Thus, a considerable proportion of the synthetic images were classified as real by clinical experts. Similarity analysis showed that the synthetic images were not copies of the real images, indicating thatcopied real images, meaning the GAN was able to generalize. However, BRISQUE score analysis indicated that synthetic images were of significantly lower quality overall than real images (P < 0.05). Comparing the rebalanced model (RB) with the baseline (R), no significant change in the average AUROC and κ was found (R-AUROC=0.86[0.85-88], RB-AUROC=0.88[0.86-0.89], R-k=0.51[0.49-0.53], and RB-k=0.52[0.50-0.54]). The synthetic data trained model (S) achieved similar performance as the baseline (S-AUROC=0.86[0.85-87], S-k=0.48[0.46-0.50]). Synthetic generation of realistic IRD FAF images is feasible. Synthetic data augmentation does not deliver improvements in classification performance. However, synthetic data alone deliver a similar performance as real data, and hence may be useful as a proxy to real data. Financial Disclosure(s): Proprietary or commercial disclosure may be found after the references.

0435 Machine-learned combination of ventilatory, hypoxic, and arousal burdens classifies daytime sleepiness better than AHI

Abstract Introduction The apnea-hypopnea index (AHI), the current severity metric used clinically for diagnosing obstructive sleep apnea (OSA), does not correlate well to daytime sleepiness measured via the Epworth Sleepiness Scale (ESS). Here, we assessed whether a machine-learned combination of possibly independent metrics across ventilatory/hypoxic/arousal domains would be better associated with ESS than the AHI using data from 3 large cohorts. Methods Polysomnography data were analyzed from The Sleep Heart Health Study (SHHS), The Multi-Ethnic Study of Atherosclerosis (MESA), and The Osteoporotic Fractures in Men (MrOS) Study. A total of N=6618 (39.9% female; age 68.7±6.6) subjects had valid data (ESS and good quality airflow/EEG/SpO2). Ventilatory burden was evaluated using a derived flow signal that utilized the sum of thoracic and abdominal effort signals for SHHS and was evaluated using the Nasal Cannula/Pressure Transducer signal for the MrOS and MESA data. Hypoxic burden was calculated as the area between the baseline and the SpO2 trace for any episode with &amp;gt;= 3% desaturation. Arousal burden was defined as the manually scored arousal index (number/hr.). Based on a cut-off of ESS(ESS&amp;gt;= 10), sleepiness was coded present or absent and was the primary outcome. Data were analyzed in two ways: using all 3 cohorts as train (70%) and test (30%), and by permutations and combinations of the 3 cohorts (70/30 split; e.g., SHHS for training, MESA and MrOS for test). Model performance metrics were the area under the receiver operating characteristic curve (AUROC) and %accuracy. For comparison, a logistic regression model using AHI3a (3% desaturation and/or EEG arousal) was fit. Results The logistic regression model (AHI3a) classified sleepiness with an AUROC of 0.51±0.07. The random forest model trained on 70% of all 3 cohorts achieved the highest AUROC of 0.88±0.07 (mean accuracy of 85.1± 2.13%), whereas the permutations and combinations of the 3 datasets resulted in an average AUROC of 0.63 ±0.12 (mean accuracy of 76.4±6.57%). Conclusion The machine-learned combination of ventilatory/hypoxic/arousal burdens classifies daytime sleepiness in OSA better than AHI3a across data from 3 large cohorts. These results suggest that OSA severity measured using machine-learned combination of ventilatory/hypoxic/arousal burdens better explains the variability in daytime sleepiness. Support (if any)

EHR foundation models improve robustness in the presence of temporal distribution shift

Temporal distribution shift negatively impacts the performance of clinical prediction models over time. Pretraining foundation models using self-supervised learning on electronic health records (EHR) may be effective in acquiring informative global patterns that can improve the robustness of task-specific models. The objective was to evaluate the utility of EHR foundation models in improving the in-distribution (ID) and out-of-distribution (OOD) performance of clinical prediction models. Transformer- and gated recurrent unit-based foundation models were pretrained on EHR of up to 1.8 M patients (382 M coded events) collected within pre-determined year groups (e.g., 2009–2012) and were subsequently used to construct patient representations for patients admitted to inpatient units. These representations were used to train logistic regression models to predict hospital mortality, long length of stay, 30-day readmission, and ICU admission. We compared our EHR foundation models with baseline logistic regression models learned on count-based representations (count-LR) in ID and OOD year groups. Performance was measured using area-under-the-receiver-operating-characteristic curve (AUROC), area-under-the-precision-recall curve, and absolute calibration error. Both transformer and recurrent-based foundation models generally showed better ID and OOD discrimination relative to count-LR and often exhibited less decay in tasks where there is observable degradation of discrimination performance (average AUROC decay of 3% for transformer-based foundation model vs. 7% for count-LR after 5–9 years). In addition, the performance and robustness of transformer-based foundation models continued to improve as pretraining set size increased. These results suggest that pretraining EHR foundation models at scale is a useful approach for developing clinical prediction models that perform well in the presence of temporal distribution shift.

Deep Learning Approach for MRI in the Classification of Anterior Talofibular Ligament Injuries.

Diagnosing anterior talofibular ligament (ATFL) injuries differs among radiologists. Further assessment of ATFL tears is valuable for clinical decision-making. To establish a deep learning method for classifying ATFL injuries based on magnetic resonance imaging (MRI). Retrospective. One thousand seventy-three patients from a single center with ankle MRI within 1 month of reference standard arthroscopy (in-group dataset), were divided into training, validation, and test sets in a ratio of 8:1:1. Additionally, 167 patients from another center were used as an independent out-group dataset. Fat-saturation proton density-weighted fast spin-echo sequence at 1.5/3.0 T. Patients were divided into normal, strain and degeneration, partial tear and complete tear groups (groups 0-3). The complete tear group was divided into five sub-groups by location and the potential avulsion fracture (groups 3.1-3.5). All images were input into AlexNet, VGG11, Small-Sample-Attention Net (SSA-Net), and SSA-Net + Weight Loss for classification. The results were compared with four radiologists with 5-30 years of experience. Model performance was evaluated by the receiver operating characteristic (ROC) curve, the area under the ROC curve (AUC), and so on. McNemar's test was used to compare performance among the different models, and between the radiologists and models. The intraclass correlation coefficient (ICC) was used to assess the reliability of the radiologists. P < 0.05 was considered statistically significant. The average AUC of AlexNet, VGG11, SAA-Net, and SSA-Net + Weight Loss was 0.95, 0.99, 0.99, 0.99 in groups 0-3 and 0.96, 0.99, 0.99, 0.99 in groups 3.1-3.5. The effect of SSA-Net + Weight Loss was similar to SSA-Net but better than AlexNet and VGG11. In the out-group test set, the AUC of SSA-Net + Weight Loss ranged from 0.89 to 0.99. The ICC of radiologists was 0.97-1.00. The effect of SSA-Net + Weight Loss was better than each radiologist in the in-group and out-group test sets. Deep learning has potential to be used for classifying ATFL injuries. SSA-Net + Weight Loss has a better diagnostic effect than radiologists with different experience levels. 4 TECHNICAL EFFICACY: Stage 2.

Multi-Class Classification Method with Feature Engineering for Predicting Hypertension with Diabetes

Machine learning–based methods are widely applied for the prediction of noncommunicable diseases (NCDs), such as hypertension, diabetes, and cardiovascular disease. However, few models have been developed for predicting hypertension with diabetes, even though these diseases generally co-occur and can cause devastating harm to patients. This paper proposes a multi-class classification method that will be able to predict hypertension with diabetes. The proposed method consists of data preprocessing, model construction and validation, and model comparison. For data preprocessing, feature engineering of corresponding data types is conducted. For model construction, several machine learning methods are applied, including Random Forest (RF), Gradient Boosting (GB), Extra Tree (ET), Decision Tree (DCT), and Support Vector Machine (SVM). The dataset used in this study consists of 17,077 records and 28 features, obtained from Phaya Mengrai Hospital, Chiang Rai, Thailand. The predictive performance of each model with and without feature engineering is compared in terms of accuracy and average area under the Receiver Operating Characteristic curve (AUC-ROC). From the comparison results, SVM with feature engineering outperformed other models based on accuracy and average AUC-ROC achieving a value of 88.39% and 93.32%, respectively. For all ensemble learning–based methods, RF performed the best in terms of both accuracy and average AUC-ROC for both with and without feature engineering. Overall, all the models performed better when feature engineering was applied.

Comparison of deep learning-based denoising methods in cardiac SPECT

BackgroundMyocardial perfusion SPECT (MPS) images often suffer from artefacts caused by low-count statistics. Poor-quality images can lead to misinterpretations of perfusion defects. Deep learning (DL)-based methods have been proposed to overcome the noise artefacts. The aim of this study was to investigate the differences among several DL denoising models.MethodsConvolution neural network (CNN), residual neural network (RES), UNET and conditional generative adversarial neural network (cGAN) were generated and trained using ordered subsets expectation maximization (OSEM) reconstructed MPS studies acquired with full, half, three-eighths and quarter acquisition time. All DL methods were compared against each other and also against images without DL-based denoising. Comparisons were made using half and quarter time acquisition data. The methods were evaluated in terms of noise level (coefficient of variation of counts, CoV), structural similarity index measure (SSIM) in the myocardium of normal patients and receiver operating characteristic (ROC) analysis of realistic artificial perfusion defects inserted into normal MPS scans. Total perfusion deficit scores were used as observer rating for the presence of a perfusion defect.ResultsAll the DL denoising methods tested provided statistically significantly lower noise level than OSEM without DL-based denoising with the same acquisition time. CoV of the myocardium counts with the different DL noising methods was on average 7% (CNN), 8% (RES), 7% (UNET) and 14% (cGAN) lower than with OSEM. All DL methods also outperformed full time OSEM without DL-based denoising in terms of noise level with both half and quarter acquisition time, but this difference was not statistically significant. cGAN had the lowest CoV of the DL methods at all noise levels. Image quality and polar map uniformity of DL-denoised images were also better than reduced acquisition time OSEM’s. SSIM of the reduced acquisition time OSEM was overall higher than with the DL methods. The defect detection performance of full time OSEM measured as area under the ROC curve (AUC) was on average 0.97. Half time OSEM, CNN, RES and UNET provided equal or nearly equal AUC. However, with quarter time data CNN, RES and UNET had an average AUC of 0.93, which was lower than full time OSEM’s AUC, but equal to quarter acquisition time OSEM. cGAN did not achieve the defect detection performance of the other DL methods. Its average AUC with half time data was 0.94 and 0.91 with quarter time data.ConclusionsDL-based denoising effectively improved noise level with slightly lower perfusion defect detection performance than full time reconstruction. cGAN achieved the lowest noise level, but at the same time the poorest defect detection performance among the studied DL methods.

Evaluate prognostic accuracy of SOFA component score for mortality among adults with sepsis by machine learning method

IntroductionSepsis has the characteristics of high incidence, high mortality of ICU patients. Early assessment of disease severity and risk stratification of death in patients with sepsis, and further targeted intervention are very important. The purpose of this study was to develop machine learning models based on sequential organ failure assessment (SOFA) components to early predict in-hospital mortality in ICU patients with sepsis and evaluate model performance.MethodsPatients admitted to ICU with sepsis diagnosis were extracted from MIMIC-IV database for retrospective analysis, and were randomly divided into training set and test set in accordance with 2:1. Six variables were included in this study, all of which were from the scores of 6 organ systems in SOFA score. The machine learning model was trained in the training set and evaluated in the validation set. Six machine learning methods including linear regression analysis, least absolute shrinkage and selection operator (LASSO), Logistic regression analysis (LR), Gaussian Naive Bayes (GNB) and support vector machines (SVM) were used to construct the death risk prediction models, and the accuracy, area under the receiver operating characteristic curve (AUROC), Decision Curve Analysis (DCA) and K-fold cross-validation were used to evaluate the prediction performance of developed models.ResultA total of 23,889 patients with sepsis were enrolled, of whom 3659 died in hospital. Three feature variables including renal system score, central nervous system score and cardio vascular system score were used to establish prediction models. The accuracy of the LR, GNB, SVM were 0.851, 0.844 and 0.862, respectively, which were better than linear regression analysis (0.123) and LASSO (0.130). The AUROCs of LR, GNB and SVM were 0.76, 0.76 and 0.67, respectively. K-fold cross validation showed that the average AUROCs of LR, GNB and SVM were 0.757 ± 0.005, 0.762 ± 0.006, 0.630 ± 0.013, respectively. For the probability threshold of 5–50%, LY and GNB models both showed positive net benefits.ConclusionThe two machine learning-based models (LR and GNB models) based on SOFA components can be used to predict in-hospital mortality of septic patients admitted to ICU.

Quantifying the daily intake of water from morning and spot urine samples; retrospective analysis of a clinical trial in volunteers

BackgroundThe hydration status can be indicated by biomarkers in the urine. However, the sensitivity and specificity of single measurements of biomarkers in morning urine and spot urine samples to quantify previous and current daily water ingestion is unclear.MethodsThe water content of food and liquid consumed by 20 volunteers (mean age 42 years) was calculated daily for two weeks. The volunteers increased their consumption of water by approximately 30% during the second week. They measured their excreted urine volume and sampled the morning urine and 24-h collections of urine for analysis of osmolality and creatinine during the first four days of both weeks (N = 157). The same biomarkers of hydration were measured in spot samples taken at every voiding on the other days (N = 762). Receiver operating characteristic (ROC) curves were used to study the ability of pre-specified ranges of biomarkers to quantify the water intake.ResultsThe biomarkers in the morning urine obtained during normal fluid intake quantified the water consumption with an average area under the ROC curve (AUC) of 0.72 for osmolality and 0.66 for creatinine. Spot urine yielded an AUC of 0.74 for osmolality and 0.70 for creatinine. The AUCs obtained for days of increased fluid intake were approximately 10% lower. Large intakes (3–4 L daily) were identified with a sensitivity of 50–80% and low intakes (< 1.5–2 L) with a sensitivity of 20–50%, while false positives occurred in approximately 10%.ConclusionBiomarkers in morning urine and spot urine samples distinguished between large and small daily water intakes. Osmolality was slightly superior to creatinine. The indications were less useful during days of increased fluid intake.