External Validation Of Model Research Articles

MOY 3 - Development and external validation of the “Global Surgical-Site Infection” (GloSSI) predictive model in adult patients undergoing abdominal surgery

Abstract Aims Identification of patients at high-risk of surgical-site infections (SSI) may allow surgeons to minimise associated morbidity. However, there are significant concerns with the methodological quality and transportability of models previously developed. We aimed to develop a novel score to predict 30-day SSI risk following gastrointestinal surgery across a global context, and externally validate against existing models. Methods This was a secondary analysis of 2 prospective international cohort studies: GlobalSurg-1 (July-November 2014) and GlobalSurg-2 (January-July 2016). Consecutive adults undergoing gastrointestinal surgery were eligible. Model development was performed in GlobalSurg-2 data, with novel and previous scores externally validated in GlobalSurg-1. The primary outcome was 30-day SSI, with two predictive techniques explored: penalised regression (LASSO) and machine learning (XGBOOST). Final model selection based on prognostic accuracy and clinical utility. Results There were 14,019 patients (SSI=12.3%) for derivation, and 8,464 (SSI=11.4%) for external validation. The LASSO model was selected due to similar discrimination to XGBoost (AUC: 0.738, 0.725-0.750 versus 0.737, 0.709-0.765), but greater explainability. The final score included six variables: country income, ASA grade, diabetes, and operative contamination, approach, and duration. Model performance remained good on external validation (AUC: 0.730, 0.715-0.744; calibration-in-the-large: -0.098, slope: 1.008), and demonstrated superior performance to external validation of all previous models. Conclusion The GloSSI score allowed accurate prediction of the risk of SSI with 6 simple variables which are routinely available at the time of surgery across global settings. This can inform use of intraoperative and postoperative interventions to modify the risk of SSI and minimise associated harm.

A Common Longitudinal Intensive Care Unit data Format (CLIF) to enable multi-institutional federated critical illness research.

Critical illness, or acute organ failure requiring life support, threatens over five million American lives annually. Electronic health record (EHR) data are a source of granular information that could generate crucial insights into the nature and optimal treatment of critical illness. However, data management, security, and standardization are barriers to large-scale critical illness EHR studies. A consortium of critical care physicians and data scientists from eight US healthcare systems developed the Common Longitudinal Intensive Care Unit (ICU) data Format (CLIF), an open-source database format that harmonizes a minimum set of ICU Data Elements for use in critical illness research. We created a pipeline to process adult ICU EHR data at each site. After development and iteration, we conducted two proof-of-concept studies with a federated research architecture: 1) an external validation of an in-hospital mortality prediction model for critically ill patients and 2) an assessment of 72-hour temperature trajectories and their association with mechanical ventilation and in-hospital mortality using group-based trajectory models. We converted longitudinal data from 94,356 critically ill patients treated in 2020-2021 (mean age 60.6 years [standard deviation 17.2], 30% Black, 7% Hispanic, 45% female) across 8 health systems and 33 hospitals into the CLIF format, The in-hospital mortality prediction model performed well in the health system where it was derived (0.81 AUC, 0.06 Brier score). Performance across CLIF consortium sites varied (AUCs: 0.74-0.83, Brier scores: 0.06-0.01), and demonstrated some degradation in predictive capability. Temperature trajectories were similar across health systems. Hypothermic and hyperthermic-slow-resolver patients consistently had the highest mortality. CLIF facilitates efficient, rigorous, and reproducible critical care research. Our federated case studies showcase CLIF's potential for disease sub-phenotyping and clinical decision-support evaluation. Future applications include pragmatic EHR-based trials, target trial emulations, foundational multi-modal AI models of critical illness, and real-time critical care quality dashboards.

Development and external validation of a risk prediction model for depression in patients with coronary heart disease

BackgroundDepression is an independent risk factor for adverse outcomes of coronary heart disease (CHD). This study aimed to develop a depression risk prediction model for CHD patients. MethodsThis study utilized data from the National Health and Nutrition Examination Survey (NHANES). In the training set, reference literature, logistic regression, LASSO regression, optimal subset algorithm, and machine learning random forest algorithm were employed to screen prediction variables, respectively. The optimal prediction model was selected based on the C-index, Net Reclassification Improvement (NRI), and Integrated Discrimination Improvement (IDI). A nomogram for the optimal prediction model was constructed. 3 external validations were performed. ResultsThe training set comprised 1375 participants, with a depressive symptoms prevalence of 15.2 %. The optimal prediction model was constructed using predictors obtained from optimal subsets algorithm (C-index = 0.774, sensitivity = 0.751, specificity = 0.685). The model includes age, gender, education, marriage, diabetes, tobacco use, antihypertensive drugs, high-density lipoprotein cholesterol (HDLC), and aspartate aminotransferase (AST). The model demonstrated consistent discrimination ability, accuracy, and clinical utility across the 3 external validations. LimitationsThe applicable population of the model is CHD patients. And the clinical benefits of interventions based on the prediction results are still unknown. ConclusionWe developed a depression risk prediction model for CHD patients, which was presented in the form of a nomogram for clinical application.

External validation of the Hong Kong Chinese non-laboratory risk models and scoring algorithm for case finding of prediabetes and diabetes mellitus in primary care.

Two Hong Kong Chinese non-laboratory-based prediabetes/diabetes mellitus (pre-DM/DM) risk models were developed using logistic regression (LR) and machine learning, respectively. We aimed to evaluate the models' validity in case finding of pre-DM/DM in a Chinese primary care (PC) population. We also evaluated the validity of a risk-scoring algorithm derived from the LR model. This was a cross-sectional external validation study on Chinese adults, without a prior DM diagnosis, who were recruited from public/private PC clinics in Hong Kong. A total of 1,237 participants completed a questionnaire on the models' predictors. Of that, 919 underwent blood glucose testing. The primary outcome was the models' and the algorithm's sensitivity in finding pre-DM/DM cases. The secondary outcomes were the models' and the algorithm's specificity, positive/negative predictive values, discrimination and calibration. The models' sensitivity were 0.70 (machine learning) and 0.72 (LR). Both showed good external discrimination (area under the receiver operating characteristic curve: machine learning 0.744, LR 0.739). The risks estimated by the models were lower than the observed incidence, indicating poor calibration. Both models were more effective among participants with lower pretest probabilities; that is, age 18-44 years. The algorithm's sensitivity was 0.77 at the cut-off score of ≥16 out of 41. This study showed the validity of the models and the algorithm for finding pre-DM/DM cases in a Chinese PC population in Hong Kong. They can facilitate more cost-effective identification of high-risk individuals for blood testing to diagnose pre-DM/DM in PC. Further studies should recalibrate the models for more precise risk estimation in PC populations.

Development and external validation of temporal fusion transformer models for continuous intraoperative blood pressure forecasting

During surgery, intraoperative hypotension is associated with postoperative morbidity and should therefore be avoided. Predicting the occurrence of hypotension in advance may allow timely interventions to prevent hypotension. Previous prediction models mostly use high-resolution waveform data, which is often not available. We utilised a novel temporal fusion transformer (TFT) algorithm to predict intraoperative blood pressure trajectories 7min in advance. We trained the model with low-resolution data (sampled every 15s) from 73,009 patients who were undergoing general anaesthesia for non-cardiothoracic surgery between January 1, 2017, and December 30, 2020, at the General Hospital of Vienna, Austria. The data set contained information on patient demographics, vital signs, medication, and ventilation. The model was evaluated using an internal (n=8113) and external test set (n=5065) obtained from the openly accessible Vital Signs Database. In the internal test set, the mean absolute error for predicting mean arterial blood pressure was 0.376 standard deviations-or 4mmHg-and 0.622 standard deviations-or 7mmHg-in the external test set. We also adapted the TFT model to binarily predict the occurrence of hypotension as defined by mean arterial blood pressure < 65mmHg in the next one, three, five, and 7min. Here, model discrimination was excellent, with a mean area under the receiver operating characteristic curve (AUROC) of 0.933 in the internal test set and 0.919 in the external test set. Our TFT model is capable of accurately forecasting intraoperative arterial blood pressure using only low-resolution data showing a low prediction error. When used for binary prediction of hypotension, we obtained excellent performance. No external funding.

Development and validation of deep learning models for identifying the brand of pedicle screws on plain spine radiographs.

In spinal revision surgery, previous pedicle screws (PS) may need to be replaced with new implants. Failure to accurately identify the brand of PS-based instrumentation preoperatively may increase the risk of perioperative complications. This study aimed to develop and validate an optimal deep learning (DL) model to identify the brand of PS-based instrumentation on plain radiographs of spine (PRS) using anteroposterior (AP) and lateral images. A total of 529 patients who received PS-based instrumentation from seven manufacturers were enrolled in this retrospective study. The postoperative PRS were gathered as ground truths. The training, validation, and testing datasets contained 338, 85, and 106 patients, respectively. YOLOv5 was used to crop out the screws' trajectory, and the EfficientNet-b0 model was used to develop single models (AP, Lateral, Merge, and Concatenated) based on the different PRS images. The ensemble models were different combinations of the single models. Primary outcomes were the models' performance in accuracy, sensitivity, precision, F1-score, kappa value, and area under the curve (AUC). Secondary outcomes were the relative performance of models versus human readers and external validation of the DL models. The Lateral model had the most stable performance among single models. The discriminative performance was improved by the ensemble method. The AP + Lateral ensemble model had the most stable performance, with an accuracy of 0.9434, F1 score of 0.9388, and AUC of 0.9834. The performance of the ensemble models was comparable to that of experienced orthopedic surgeons and superior to that of inexperienced orthopedic surgeons. External validation revealed that the Lat + Concat ensemble model had the best accuracy (0.9412). The DL models demonstrated stable performance in identifying the brand of PS-based instrumentation based on AP and/or lateral images of PRS, which may assist orthopedic spine surgeons in preoperative revision planning in clinical practice.

Energy expenditure prediction in preschool children: a machine learning approach using accelerometry and external validation

Objective.This study aimed to develop convolutional neural networks (CNNs) models to predict the energy expenditure (EE) of children from raw accelerometer data. Additionally, this study sought to external validation of the CNN models in addition to the linear regression (LM), random forest (RF), and full connected neural network (FcNN) models published in Steenbocket al(2019J. Meas. Phys. Behav.294-102).Approach.Included in this study were 41 German children (3.0-6.99 years) for the training and internal validation who were equipped with GENEActiv, GT3X+, and activPAL accelerometers. The external validation dataset consisted of 39 Canadian children (3.0-5.99 years) that were equipped with OPAL, GT9X, GENEActiv, and GT3X+ accelerometers. EE was recorded simultaneously in both datasets using a portable metabolic unit. The protocols consisted of a semi-structured activities ranging from low to high intensities. The root mean square error (RMSE) values were calculated and used to evaluate model performances.Main results.(1) The CNNs outperformed the LM (13.17%-23.81% lower mean RMSE values), FcNN (8.13%-27.27% lower RMSE values) and the RF models (3.59%-18.84% lower RMSE values) in the internal dataset. (2) In contrast, it was found that when applied to the external Canadian dataset, the CNN models had consistently higher RMSE values compared to the LM, FcNN, and RF.Significance.Although CNNs can enhance EE prediction accuracy, their ability to generalize to new datasets and accelerometer brands/models, is more limited compared to LM, RF, and FcNN models.

1986: External validation of a deep learning prostate MR auto-contouring model

1986: External validation of a deep learning prostate MR auto-contouring model

Single-center external validation and reconstruction of multiple predictive models for skip lateral lymph node metastasis in papillary thyroid carcinoma.

The unique metastatic pattern of skip lateral lymph node metastasis (SLLNM) in PTC patients may lead to missed diagnosis of lateral cervical metastatic lymph nodes. Therefore, many different SLLNM prediction models were constructed. In this study, partially eligible models (Hu 2020, Wang 2020, and Zhao 2023 nomograms) were selected for external validation, and then new variables were incorporated for model reconstruction to extend clinical applicability. 576 PTC patients from our center were selected to evaluate the performance of the three nomograms using the receiver operating characteristic curve (ROC), calibration curves, and decision curve analyses (DCA). Three new variables were added to calibrate the model, including assessment of LN status on ultrasound (US-SLLNM), the distance from the tumor to the capsule (Capsular distance), and the number of central lymph node dissections (CLND number). Univariate and multivariate logistic regression analyses were used to screen independent predictors to reconstruct the model, and 1000 Bootstrap internal validations were performed. SLLNM were present in 69/576 patients (12.0%). In external validation, the area under the ROC curves (AUCs) for Hu 2020, Wang 2020, and Zhao 2023 nomograms were 0.695 (95% CI:0.633-0.766), 0.792 (95% CI=0.73-0.845), and 0.769 (95% CI:0.713-0.824), respectively. The calibration curves for the three models were overall poorly fitted; DCA showed some net clinical benefit. Model differentiation and net clinical benefit improved by adding three new variables. Based on multivariate analysis, female, age, and maximum tumor diameter ≤ 10 mm, located at the upper pole, Capsular distance < 0mm, US-SLLNM, CLND number ≤ 5 were identified as independent predictors of SLLNM and were used to construct the new model. After 1000 Bootstrap internal validations, the mean AUC of the model was 0.870 (95% CI:0.839-0.901), the calibration curve was close to the ideal curve, and the net clinical benefit was significant. Overall, these nomograms were well differentiated and provided some net clinical benefit, but with varying degrees of underestimation or overestimation of the actual risk and high false-negative rates. New dynamic nomogram was constructed based on the addition of new variables and larger samples, showing better performance.

Derivation and external validation of predictive models for invasive mechanical ventilation in intensive care unit patients with COVID-19

BackgroundThis study aimed to develop prognostic models for predicting the need for invasive mechanical ventilation (IMV) in intensive care unit (ICU) patients with COVID-19 and compare their performance with the Respiratory rate-OXygenation (ROX) index.MethodsA retrospective cohort study was conducted using data collected between March 2020 and August 2021 at three hospitals in Rio de Janeiro, Brazil. ICU patients aged 18 years and older with a diagnosis of COVID-19 were screened. The exclusion criteria were patients who received IMV within the first 24 h of ICU admission, pregnancy, clinical decision for minimal end-of-life care and missing primary outcome data. Clinical and laboratory variables were collected. Multiple logistic regression analysis was performed to select predictor variables. Models were based on the lowest Akaike Information Criteria (AIC) and lowest AIC with significant p values. Assessment of predictive performance was done for discrimination and calibration. Areas under the curves (AUC)s were compared using DeLong’s algorithm. Models were validated externally using an international database.ResultsOf 656 patients screened, 346 patients were included; 155 required IMV (44.8%), 191 did not (55.2%), and 207 patients were male (59.8%). According to the lowest AIC, arterial hypertension, diabetes mellitus, obesity, Sequential Organ Failure Assessment (SOFA) score, heart rate, respiratory rate, peripheral oxygen saturation (SpO2), temperature, respiratory effort signals, and leukocytes were identified as predictors of IMV at hospital admission. According to AIC with significant p values, SOFA score, SpO2, and respiratory effort signals were the best predictors of IMV; odds ratios (95% confidence interval): 1.46 (1.07–2.05), 0.81 (0.72–0.90), 9.13 (3.29–28.67), respectively. The ROX index at admission was lower in the IMV group than in the non-IMV group (7.3 [5.2–9.8] versus 9.6 [6.8–12.9], p < 0.001, respectively). In the external validation population, the area under the curve (AUC) of the ROX index was 0.683 (accuracy 63%), the AIC model showed an AUC of 0.703 (accuracy 69%), and the lowest AIC model with significant p values had an AUC of 0.725 (accuracy 79%).ConclusionsIn the development population of ICU patients with COVID-19, SOFA score, SpO2, and respiratory effort signals predicted the need for IMV better than the ROX index. In the external validation population, although the AUCs did not differ significantly, the accuracy was higher when using SOFA score, SpO2, and respiratory effort signals compared to the ROX index. This suggests that these variables may be more useful in predicting the need for IMV in ICU patients with COVID-19.ClinicalTrials.gov identifier:NCT05663528.

External validation of and improvement upon a model for the prediction of placenta accreta spectrum severity using prospectively collected multicenter ultrasound data.

This study aimed to validate the Sargent risk stratification algorithm for the prediction of placenta accreta spectrum (PAS) severity using data collected from multiple centers and using the multicenter data to improve the model. We conducted a multicenter analysis using data collected for the IS-PAS database. The Sargent model's effectiveness in distinguishing between abnormally adherent placenta (FIGO grade 1) and abnormally invasive placenta (FIGO grades 2 and 3) was evaluated. A new model was developed using multicenter data from the IS-PAS database. The database included 315 cases of suspected PAS, of which 226 had fully documented standardized ultrasound signs. The final diagnosis was normal placentation in 5, abnormally adherent placenta/FIGO grade 1 in 43, and abnormally invasive placenta/FIGO grades 2 and 3 in 178. The external validation of the Sargent model revealed moderate predictive accuracy in a multicenter setting (C-index 0.68), compared to its higher accuracy in a single-center context (C-index 0.90). The newly developed model achieved a C-index of 0.74. The study underscores the difficulty in developing universally applicable PAS prediction models. While models like that of Sargent etal. show promise, their reproducibility varies across settings, likely due to the interpretation of the ultrasound signs. The findings support the need for updating the current ultrasound descriptors and for the development of any new predictive models to use data collected by different operators in multiple clinical settings.

Population Pharmacokinetic of Vancomycin Administered by Continuous Infusion in Critically Ill Patients

Introduction: Administration of vancomycin dose by continuous infusion (CI) according to population pharmacokinetic (Pop Pk) models is highly recommended in critically ill patients who exhibit pathophysiological changes. Objective: The objective of this study was to develop and validate a Pop Pk model of vancomycin administered by CI in critically ill patients with normal and impaired renal functions. Methods: The Pop Pk study was performed using a nonparametric approach (Pmetrics*). The influence of covariates (gender, age, weight, height, and creatinine clearance [Cr-Cl]) was tested on the model’s Pk parameters. The performance of the final model was assessed using an external dataset. Results: A one-compartment model (volume of distribution [Vd], elimination from compartment [Ke]) was found to show a good prediction performance. The influence of covariates has shown that age and Cr-Cl affected significantly Vd and Ke, respectively. The distribution of simulated vancomycin clearance (CLv) according to different renal function levels showed a negative correlation between CLv and the severity of the renal impairment. The internal validation of the final model showed that the plot of individual-predicted concentration versus observed concentration resulted in r2 = 0.86 in the final model. The external validation of the final model showed an acceptable predictive performance. Conclusion: We developed a Pop Pk model for vancomycin administered by CI in critically ill patients. A significant impact of Cr-Cl and different stages of renal failure on CLv has been demonstrated. The establishment of an individualized proposal dose based on this model may be helpful to achieve the target range which is critical in optimizing the efficacy and safety of this antibiotic.

External validation of models to estimate gestational age in the second and third trimester using ultrasound: A prospective multicentre observational study.

Accurate assessment of gestational age (GA) is important at both individual and population levels. The most accurate way to estimate GA in women who book late in pregnancy is unknown. The aim of this study was to externally validate the accuracy of equations for GA estimation in late pregnancy and to identify the best equation for estimating GA in women who do not receive an ultrasound scan until the second or third trimester. This was a prospective, observational cross-sectional study. 57 prenatal care centres, France. Women with a singleton pregnancy and a previous 11-14-week dating scan that gave the observed GA were recruited over an 8-week period. They underwent a standardised ultrasound examination at one time point during the pregnancy (15-43 weeks), measuring 12 foetal biometric parameters that have previously been identified as useful for GA estimation. A total of 189 equations that estimate GA based on foetal biometry were examined and compared with GA estimation based on foetal CRL. Comparisons between the observed GA and the estimated GA were made using R2, calibration slope and intercept. RMSE, mean difference and 95% range of error were also calculated. A total of 2741 pregnant women were examined. After exclusions, 2339 participants were included. In the 20 best performing equations, the intercept ranged from -0.22 to 0.30, the calibration slope from 0.96 to 1.03 and the RSME from 0.67 to 0.87. Overall, multiparameter models outperformed single-parameter models. Both the 95% range of error and mean difference increased with gestation. Commonly used models based on measurement of the head circumference alone were not amongst the best performing models and were associated with higher 95% error and mean difference. We provide strong evidence that GA-specific equations based on multiparameter models should be used to estimate GA in late pregnancy. However, as all methods of GA assessment in late pregnancy are associated with large prediction intervals, efforts to improve access to early antenatal ultrasound must remain a priority. The proposal for this study and the corresponding methodological review was registered on PROSPERO international register of systematic reviews (registration number: CRD4201913776).

The Derivation and External Validation of a Fibrosis Risk Model for Colorectal Tumours Undergoing Endoscopic Submucosal Dissection.

Background: Endoscopic submucosal dissection (ESD) is an advanced technique that can become more challenging in the presence of submucosal fibrosis. Predicting the grade of fibrosis is important in order to identify technically difficult ESD. Aims and Methods: Our study aimed to derive and validate a prediction model to determine the preoperative degree of submucosal fibrosis in colorectal tumours undergoing ESD. A predictive model was developed to derive the probability of an increasing submucosal fibrosis in the derivation cohort and then externally validated. Results: 309 patients (age: 68 ± 10.9 years) underwent colorectal ESD between January 2016 and June 2020. F0, F1, and F2 fibroses were reported in 196 (63.4%), 70 (22.6%), and 43 (13.9%) cases, respectively. R0 resection was found in 266 (87%) lesions. At multivariable analysis in the derivation cohort, lesion morphology (OR = 0.37 and CI = 0.14-0.97 for LST-NG vs. 0-Is; OR = 0.29 and CI = 0.1-0.87 for the LST mixed type vs. 0-Is; and OR = 0.32 and CI = 0.1-1.03 for LST-G vs. 0-Is) and increasing size (OR = 1.02 and CI = 1.01-1.04 for a 1 mm increase) were significantly associated with an increasing degree of fibrosis. The model had fair discriminating ability in the derivation group (AUROC = 0.61 and CI = 0.52-0.69 for F1-F2 vs. F0 fibroses; AUROC = 0.61 and CI = 0.45-0.77 for F2 vs. F0-F1 fibroses) and in the validation group (AUROC = 0.71 and CI = 0.59-0.83 for F1-F2 vs. F0 fibroses; AUROC = 0.65 and CI = 0.52-0.77 for F2 vs. F0-F1 fibroses). Conclusions: Our findings introduce a new tool for the stratification of ESD technical difficulty based on lesion size and morphological characteristics which could become crucial during the procedure's planning process.

Establishment and evaluation of a clinical prediction model for cognitive impairment in patients with cerebral small vessel disease

BackgroundThere are currently no effective prediction methods for evaluating the occurrence of cognitive impairment in patients with cerebral small vessel disease (CSVD).AimsTo investigate the risk factors for cognitive dysfunction in patients with CSVD and to construct a risk prediction model.MethodsA retrospective study was conducted on 227 patients with CSVD. All patients were assessed by brain magnetic resonance imaging (MRI), and the Montreal Cognitive Assessment (MoCA) was used to assess cognitive status. In addition, the patient’s medical records were also recorded. The clinical data were divided into a normal cognitive function group and a cognitive impairment group. A MoCA score < 26 (an additional 1 point for education < 12 years) is defined as cognitive dysfunction.ResultsA total of 227 patients (mean age 66.7 ± 6.99 years) with CSVD were included in this study, of whom 68.7% were male and 100 patients (44.1%) developed cognitive impairment. Age (OR = 1.070; 95% CI = 1.015 ~ 1.128, p < 0.05), hypertension (OR = 2.863; 95% CI = 1.438 ~ 5.699, p < 0.05), homocysteine(HCY) (OR = 1.065; 95% CI = 1.005 ~ 1.127, p < 0.05), lacunar infarct score(Lac_score) (OR = 2.732; 95% CI = 1.094 ~ 6.825, P < 0.05), and CSVD total burden (CSVD_score) (OR = 3.823; 95% CI = 1.496 ~ 9.768, P < 0.05) were found to be independent risk factors for cognitive decline in the present study. The above 5 variables were used to construct a nomogram, and the model was internally validated by using bootstrapping with a C-index of 0.839. The external model validation C-index was 0.867.ConclusionsThe nomogram model based on brain MR images and clinical data helps in individualizing the probability of cognitive impairment progression in patients with CSVD.

External validation of a predictive model for aged-related skin tears in Fitzpatrick skin types I-IV cohort.

Older Australians are at increased risk of skin tears with the risk not always recognised or the injury able to be prevented. This study externally validated Rayner et al. (2019) Skin Tear Risk Prediction Model in an independent aged cohort with a Fitzpatrick skin types I-IV from across multiple residential-care sites, over a 6-month period. A total of 362 individuals aged between 65 and 102.5 years completed the study. In all, 165-residents sustained one or more skin tears. Logistic regression analysis was conducted of the five variables (gender, previous history of skin tears, previous history of falls, purpura and solar elastosis) identified in the skin tear model. The skin tear model provided 'good' to nearly 'very good discrimination' for correctly classifying residents at-risk or not-at-risk (area under the curve of 0.799 [95% confidence interval, CI: 0.75-0.84]). The skin tear model correctly predicted 75.8% (sensitivity) of participants with skin tears and 71.6% (specificity) of residents without skin tears. The model demonstrated it could work as a screening tool to identify older individuals at risk of skin tears and would benefit clinical practice as it was easy to use, was reproducible, and had good accuracy across aged-care residents with a Fitzpatrick skin type I-IV.

External validation of the PIERS-AI model in a UK cohort of women with pre-eclampsia

External validation of the PIERS-AI model in a UK cohort of women with pre-eclampsia

Risk of serious adverse events after primary shoulder replacement: development and external validation of a prediction model using linked national data from England and Denmark

Despite a rising rate of serious medical complications after shoulder replacement surgery, there are no prediction models in widespread use to guide surgeons in identifying patients at high risk and to provide patients with personalised risk estimates to support shared decision making. Our aim was to develop and externally validate a prediction model for serious adverse events within 90 days of primary shoulder replacement surgery. Linked data from the National Joint Registry, National Health Service Hospital Episode Statistics Admitted Patient Care of England, and Civil Registration Mortality databases and Danish Shoulder Arthroplasty Registry and National Patient Register were used for our modelling study. Patients aged 18-100 years who had a primary shoulder replacement between April 1, 2012, and Oct 2, 2020, in England, and April 1, 2012, and Oct 2, 2018, in Denmark, were included. We developed a multivariable logistic regression model using the English dataset to predict the risk of 90-day serious adverse events, which were defined as medical complications requiring admission to hospital and all-cause death. We undertook internal validation using bootstrapping, and internal-external cross-validation across different geographical regions of England. The English model was externally validated on the Danish dataset. Data for 40 631 patients undergoing primary shoulder replacement (mean age 72·5 years [SD 9·9]; 28 709 [70·7%] women and 11 922 [29·3%] men) were used for model development, of whom 2270 (5·6%) had a 90-day serious adverse event. On internal validation, the model had a C-statistic of 0·717 (95% CI 0·707-0·728) and was well calibrated. Internal-external cross-validation showed consistent model performance across all regions in England. Upon external validation on the Danish dataset (n=6653; mean age 70·5 years [SD 10·3]; 4503 [67·7%] women and 2150 [32·3%] men), the model had a C-statistic of 0·750 (95% CI 0·723-0·776). Decision curve analysis showed clinical utility, with net benefit across all risk thresholds. This externally validated prediction model uses commonly available clinical variables to accurately predict the risk of serious medical complications after primary shoulder replacement surgery. The model is generalisable and applicable to most patients in need of a shoulder replacement. Its use offers support to clinicians and could inform and empower patients in the shared decision-making process. National Institute for Health and Care Research and the Department of Orthopaedic Surgery, Herlev and Gentofte Hospital, Denmark.

An ensemble machine learning model for predicting one-year mortality in elderly coronary heart disease patients with anemia

ObjectiveThis study was designed to develop and validate a robust predictive model for one-year mortality in elderly coronary heart disease (CHD) patients with anemia using machine learning methods.MethodsDemographics, tests, comorbidities, and drugs were collected for a cohort of 974 elderly patients with CHD. A prospective analysis was performed to evaluate predictive performances of the developed models. External validation of models was performed in a series of 112 elderly CHD patients with anemia.ResultsThe overall one-year mortality was 43.6%. Risk factors included heart rate, chronic heart failure, tachycardia and β receptor blockers. Protective factors included hemoglobin, albumin, high density lipoprotein cholesterol, estimated glomerular filtration rate (eGFR), left ventricular ejection fraction (LVEF), aspirin, clopidogrel, calcium channel blockers, angiotensin converting enzyme inhibitors (ACEIs)/angiotensin receptor blockers (ARBs), and statins. Compared with other algorithms, an ensemble machine learning model performed the best with area under the curve (95% confidence interval) being 0.828 (0.805–0.870) and Brier score being 0.170. Calibration and density curves further confirmed favorable predicted probability and discriminative ability of an ensemble machine learning model. External validation of Ensemble Model also exhibited good performance with area under the curve (95% confidence interval) being 0.825 (0.734–0.916) and Brier score being 0.185. Patients in the high-risk group had more than six-fold probability of one-year mortality compared with those in the low-risk group (P < 0.001). Shaley Additive exPlanation identified the top five risk factors that associated with one-year mortality were hemoglobin, albumin, eGFR, LVEF, and ACEIs/ARBs.ConclusionsThis model identifies key risk factors and protective factors, providing valuable insights for improving risk assessment, informing clinical decision-making and performing targeted interventions. It outperforms other algorithms with predictive performance and provides significant opportunities for personalized risk mitigation strategies, with clinical implications for improving patient care.

Development and external validation of a predictive model for type 2 diabetic retinopathy

Diabetes retinopathy (DR) is a critical clinical disease with that causes irreversible visual damage in adults, and may even lead to permanent blindness in serious cases. Early identification and treatment of DR is critical. Our aim was to train and externally validate a prediction nomogram for early prediction of DR. 2381 patients with type 2 diabetes mellitus (T2DM) were retrospective study from the First Affiliated Hospital of Xinjiang Medical University in Xinjiang, China, hospitalised between Jan 1, 2019 and Jun 30, 2022. 962 patients with T2DM from the Suzhou BenQ Hospital in Jiangsu, China hospitalised between Jul 1, 2020 to Jun 30, 2022 were considered for external validation. The least absolute shrinkage and selection operator (LASSO) and multivariate logistic regression was performed to identify independent predictors and establish a nomogram to predict the occurrence of DR. The performance of the nomogram was evaluated using a receiver operating characteristic curve (ROC), a calibration curve, and decision curve analysis (DCA). Neutrophil, 25-hydroxyvitamin D3 [25(OH)D3], Duration of T2DM, hemoglobin A1c (HbA1c), and Apolipoprotein A1 (ApoA1) were used to establish a nomogram model for predicting the risk of DR. In the development and external validation groups, the areas under the curve of the nomogram constructed from the above five factors were 0.834 (95%CI 0.820–0.849) and 0.851 (95%CI 0.829–0.874), respectively. The nomogram demonstrated excellent performance in the calibration curve and DCA. This research has developed and externally verified that the nomograph model shows a good predictive ability in assessing DR risk in people with type 2 diabetes. The application of this model will help clinicians to intervene early, thus effectively reducing the incidence rate and mortality of DR in the future, and has far-reaching significance in improving the long-term health prognosis of diabetes patients.