Validation Of Risk Prediction Model Research Articles

Development and validation of risk prediction model for refeeding syndrome in neurocritical patients.

The incidence of refeeding syndrome (RFS) in critically ill patients is high, which is detrimental to their prognoses. However, the current status and risk factors for the occurrence of RFS in neurocritical patients remain unclear. Elucidating these aspects may provide a theoretical basis for screening populations at high risk of RFS. A total of 357 patients from January 2021 to May 2022 in a neurosurgery ICU of a tertiary hospital in China were included using convenience sampling. Patients were divided into RFS and non-RFS groups, based on the occurrence of refeeding-associated hypophosphatemia. Risk factors for RFS were determined using univariate and logistic regression analyses, and a risk prediction model for RFS in neurocritical patients was developed. The Hosmer-Lemeshow test was used to determine the goodness of fit of the model, and the receiver operator characteristic curve was used to examine its discriminant validity. The incidence of RFS in neurocritical patients receiving enteral nutrition was 28.57%. Logistic regression analyses showed that history of alcoholism, fasting hours, Acute Physiology and Chronic Health Evaluation II (APACHE II) scores, Sequential Organ Failure Assessment (SOFA) scores, low serum albumin, and low baseline serum potassium were risk factors of RFS in neurocritical patients (p < 0.05). The Hosmer-Lemeshow test showed p = 0.616, and the area under the ROC curve was 0.791 (95% confidence interval: 0.745-0.832). The optimal critical value was 0.299, the sensitivity was 74.4%, the specificity was 77.7%, and the Youden index was 0.492. The incidence of RFS in neurocritical patients was high, and the risk factors were diverse. The risk prediction model in this study had good predictive effects and clinical utility, which may provide a reference for assessing and screening for RFS risk in neurocritical patients.

Development and validation of risk prediction models for new-onset type 2 diabetes in adults with impaired fasting glucose

AimsTo develop and validate sex-specific risk prediction models based on easily obtainable clinical data for predicting 5-year risk of type 2 diabetes (T2D) among individuals with impaired fasting glucose (IFG), and generate practical tools for public use. MethodsThe data used for model training and internal validation came from a large prospective cohort (N = 18,384). Two independent cohorts were used for external validation. A two-step approach was applied to screen variables. Coefficient-based models were constructed by multivariate Cox regression analyses, and score-based models were subsequently generated. The predictive power was evaluated by the area under the curve (AUC). ResultsDuring a median follow-up of 7.55 years, 5697 new-onset T2D cases were identified. Predictor variables included age, body mass index, waist circumference, diastolic blood pressure, triglycerides, fasting plasma glucose, and fatty liver. The proposed models outperformed five existing models. In internal validation, the AUCs of the coefficient-based models were 0.741 (95% CI 0.723–0.760) for men and 0.762 (95% CI 0.720–0.802) for women. External validation yielded comparable prediction performance. We finally constructed a risk scoring system and a web calculator. ConclusionsThe risk prediction models and derived tools had well-validated performance to predict the 5-year risk of T2D in IFG adults.

Development and validation of risk prediction model for identifying 30-day frailty in older inpatients with undernutrition: A multicenter cohort study.

To develop and externally validate a frailty prediction model integrating physical factors, psychological variables and routine laboratory test parameters to predict the 30-day frailty risk in older adults with undernutrition. Based on an ongoing survey of geriatrics syndrome in elder adults across China (SGSE), this prognostic study identified the putative prognostic indicators for predicting the 30-day frailty risk of older adults with undernutrition. Using multivariable logistic regression analysis with backward elimination, the predictive model was subjected to internal (bootstrap) and external validation, and its calibration was evaluated by the calibration slope and its C statistic discriminative ability. The model derivation and model validation cohorts were collected between October 2018 and February 2019 from a prospective, large-scale cohort study of hospitalized older adults in tertiary hospitals in China. The modeling derivation cohort data (n = 2,194) were based on the SGSE data comprising southwest Sichuan Province, northern Beijing municipality, northwest Qinghai Province, northeast Heilongjiang Province, and eastern Zhejiang Province, with SGSE data from Hubei Province used to externally validate the model (validation cohort, n = 648). The incidence of frailty in the older undernutrition derivation cohort was 13.54% and 13.43% in the validation cohort. The final model developed to estimate the individual predicted risk of 30-day frailty was presented as a regression formula: predicted risk of 30-day frailty = [1/(1+e-riskscore )], where riskscore = -0.106 + 0.034 × age + 0.796 × sex -0.361 × vision dysfunction + 0.373 × hearing dysfunction + 0.408 × urination dysfunction - 0.012 × ADL + 0.064 × depression - 0.139 × nutritional status - 0.007 × hemoglobin - 0.034 × serum albumin - 0.012 × (male: ADL). Area under the curve (AUC) of 0.71 in the derivation cohort, and discrimination of the model were similar in both cohorts, with a C statistic of nearly 0.7, with excellent calibration of observed and predicted risks. A new prediction model that quantifies the absolute risk of frailty of older patients suffering from undernutrition was developed and externally validated. Based on physical, psychological, and biological variables, the model provides an important assessment tool to provide different healthcare needs at different times for undernutrition frailty patients. Chinese Clinical Trial Registry [ChiCTR1800017682].

Improving risk prediction model quality in the critically ill: data linkage study

Background A previous National Institute for Health and Care Research study [Harrison DA, Ferrando-Vivas P, Shahin J, Rowan KM. Ensuring comparisons of health-care providers are fair: development and validation of risk prediction models for critically ill patients. Health Serv Deliv Res 2015;3(41)] identified the need for more research to understand risk factors and consequences of critical care and subsequent outcomes. Objectives First, to improve risk models for adult general critical care by developing models for mortality at fixed time points and time-to-event outcomes, end-stage renal disease, type 2 diabetes, health-care utilisation and costs. Second, to improve risk models for cardiothoracic critical care by enhancing risk factor data and developing models for longer-term mortality. Third, to improve risk models for in-hospital cardiac arrest by enhancing risk factor data and developing models for longer-term mortality and critical care utilisation. Design Risk modelling study linking existing data. Setting NHS adult critical care units and acute hospitals in England. Participants Patients admitted to an adult critical care unit or experiencing an in-hospital cardiac arrest. Interventions None. Main outcome measures Mortality at hospital discharge, 30 days, 90 days and 1 year following critical care unit admission; mortality at 1 year following discharge from acute hospital; new diagnosis of end-stage renal disease or type 2 diabetes; hospital resource use and costs; return of spontaneous circulation sustained for &gt; 20 minutes; survival to hospital discharge and 1 year; and length of stay in critical care following in-hospital cardiac arrest. Data sources Case Mix Programme, National Cardiac Arrest Audit, UK Renal Registry, National Diabetes Audit, National Adult Cardiac Surgery Audit, Hospital Episode Statistics and Office for National Statistics. Results Data were linked for 965,576 critical care admissions between 1 April 2009 and 31 March 2016, and 83,939 in-hospital cardiac arrests between 1 April 2011 and 31 March 2016. For admissions to adult critical care units, models for 30-day mortality had similar predictors and performance to those for hospital mortality and did not reduce heterogeneity. Models for longer-term outcomes reflected increasing importance of chronic over acute predictors. New models for end-stage renal disease and diabetes will allow benchmarking of critical care units against these important outcomes and identification of patients requiring enhanced follow-up. The strongest predictors of health-care costs were prior hospitalisation, prior dependency and chronic conditions. Adding pre- and intra-operative risk factors to models for cardiothoracic critical care gave little improvement in performance. Adding comorbidities to models for in-hospital cardiac arrest provided modest improvements but were of greater importance for longer-term outcomes. Limitations Delays in obtaining linked data resulted in the data used being 5 years old at the point of publication: models will already require recalibration. Conclusions Data linkage provided enhancements to the risk models underpinning national clinical audits in the form of additional predictors and novel outcomes measures. The new models developed in this report may assist in providing objective estimates of potential outcomes to patients and their families. Future work (1) Develop and test care pathways for recovery following critical illness targeted at those with the greatest need; (2) explore other relevant data sources for longer-term outcomes; (3) widen data linkage for resource use and costs to primary care, outpatient and emergency department data. Study registration This study is registered as NCT02454257. Funding details This project was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme and will be published in full in Health and Social Care Delivery Research; Vol. 10, No. 39. See the NIHR Journals Library website for further project information.

Development and Validation of an Integrated Suite of Prediction Models for All-Cause 30-Day Readmissions of Children and Adolescents Aged 0 to 18 Years

Readmission is often considered a hospital quality measure, yet no validated risk prediction models exist for children. To develop and validate a tool identifying patients before hospital discharge who are at risk for subsequent readmission, applicable to all ages. This population-based prognostic analysis used electronic health record-derived data from a freestanding children's hospital from January 1, 2016, to December 31, 2019. All-cause 30-day readmission was modeled using 3 years of discharge data. Data were analyzed from June 1 to November 30, 2021. Three models were derived as a complementary suite to include (1) children 6 months or older with 1 or more prior hospitalizations within the last 6 months (recent admission model [RAM]), (2) children 6 months or older with no prior hospitalizations in the last 6 months (new admission model [NAM]), and (3) children younger than 6 months (young infant model [YIM]). Generalized mixed linear models were used for all analyses. Models were validated using an additional year of discharges. The derivation set contained 29 988 patients with 48 019 hospitalizations; 50.1% of these admissions were for children younger than 5 years and 54.7% were boys. In the derivation set, 4878 of 13 490 admissions (36.2%) in the RAM cohort, 2044 of 27 531 (7.4%) in the NAM cohort, and 855 of 6998 (12.2%) in the YIM cohort were followed within 30 days by a readmission. In the RAM cohort, prior utilization, current or prior procedures indicative of severity of illness (transfusion, ventilation, or central venous catheter), commercial insurance, and prolonged length of stay (LOS) were associated with readmission. In the NAM cohort, procedures, prolonged LOS, and emergency department visit in the past 6 months were associated with readmission. In the YIM cohort, LOS, prior visits, and critical procedures were associated with readmission. The area under the receiver operating characteristics curve was 83.1 (95% CI, 82.4-83.8) for the RAM cohort, 76.1 (95% CI, 75.0-77.2) for the NAM cohort, and 80.3 (95% CI, 78.8-81.9) for the YIM cohort. In this prognostic study, the suite of 3 prediction models had acceptable to excellent discrimination for children. These models may allow future improvements in tailored discharge preparedness to prevent high-risk readmissions.

AutoScore-Ordinal: an interpretable machine learning framework for generating scoring models for ordinal outcomes

BackgroundRisk prediction models are useful tools in clinical decision-making which help with risk stratification and resource allocations and may lead to a better health care for patients. AutoScore is a machine learning–based automatic clinical score generator for binary outcomes. This study aims to expand the AutoScore framework to provide a tool for interpretable risk prediction for ordinal outcomes.MethodsThe AutoScore-Ordinal framework is generated using the same 6 modules of the original AutoScore algorithm including variable ranking, variable transformation, score derivation (from proportional odds models), model selection, score fine-tuning, and model evaluation. To illustrate the AutoScore-Ordinal performance, the method was conducted on electronic health records data from the emergency department at Singapore General Hospital over 2008 to 2017. The model was trained on 70% of the data, validated on 10% and tested on the remaining 20%.ResultsThis study included 445,989 inpatient cases, where the distribution of the ordinal outcome was 80.7% alive without 30-day readmission, 12.5% alive with 30-day readmission, and 6.8% died inpatient or by day 30 post discharge. Two point-based risk prediction models were developed using two sets of 8 predictor variables identified by the flexible variable selection procedure. The two models indicated reasonably good performance measured by mean area under the receiver operating characteristic curve (0.758 and 0.793) and generalized c-index (0.737 and 0.760), which were comparable to alternative models.ConclusionAutoScore-Ordinal provides an automated and easy-to-use framework for development and validation of risk prediction models for ordinal outcomes, which can systematically identify potential predictors from high-dimensional data.

Development and Validation of Risk Prediction Models for Gestational Diabetes Mellitus Using Four Different Methods.

Gestational diabetes mellitus (GDM), a common perinatal disease, is related to increased risks of maternal and neonatal adverse perinatal outcomes. We aimed to establish GDM risk prediction models that can be widely used in the first trimester using four different methods, including a score-scaled model derived from a meta-analysis using 42 studies, a logistic regression model, and two machine learning models (decision tree and random forest algorithms). The score-scaled model (seven variables) was established via a meta-analysis and a stratified cohort of 1075 Chinese pregnant women from the Northwest Women's and Children's Hospital (NWCH) and showed an area under the curve (AUC) of 0.772. The logistic regression model (seven variables) was established and validated using the above cohort and showed AUCs of 0.799 and 0.834 for the training and validation sets, respectively. Another two models were established using the decision tree (DT) and random forest (RF) algorithms and showed corresponding AUCs of 0.825 and 0.823 for the training set, and 0.816 and 0.827 for the validation set. The validation of the developed models suggested good performance in a cohort derived from another period. The score-scaled GDM prediction model, the logistic regression GDM prediction model, and the two machine learning GDM prediction models could be employed to identify pregnant women with a high risk of GDM using common clinical indicators, and interventions can be sought promptly.

Validation of risk prediction models applied to longitudinal electronic health record data for the prediction of major cardiovascular events in the presence of data shifts.

Deep learning has dominated predictive modelling across different fields, but in medicine it has been met with mixed reception. In clinical practice, simple, statistical models and risk scores continue to inform cardiovascular disease risk predictions. This is due in part to the knowledge gap about how deep learning models perform in practice when they are subject to dynamic data shifts; a key criterion that common internal validation procedures do not address. We evaluated the performance of a novel deep learning model, BEHRT, under data shifts and compared it with several ML-based and established risk models. Using linked electronic health records of 1.1 million patients across England aged at least 35 years between 1985 and 2015, we replicated three established statistical models for predicting 5-year risk of incident heart failure, stroke, and coronary heart disease. The results were compared with a widely accepted machine learning model (random forests), and a novel deep learning model (BEHRT). In addition to internal validation, we investigated how data shifts affect model discrimination and calibration. To this end, we tested the models on cohorts from (i) distinct geographical regions; (ii) different periods. Using internal validation, the deep learning models substantially outperformed the best statistical models by 6%, 8%, and 11% in heart failure, stroke, and coronary heart disease, respectively, in terms of the area under the receiver operating characteristic curve. The performance of all models declined as a result of data shifts; despite this, the deep learning models maintained the best performance in all risk prediction tasks. Updating the model with the latest information can improve discrimination but if the prior distribution changes, the model may remain miscalibrated.

Validation of cardiovascular risk prediction in Type 2 diabetes through federated cohorts in Europe

Abstract Background The SCORE2 risk model has been recommended for cardiovascular risk assessment in individuals aged over 40 years without diabetes in 4 defined risk regions of Europe. We aimed to validate a novel SCORE2-DM model in Type 2 diabetes with additional risk factors mainly based on UK datasets, using federated databases from the EUBIROD network. Methods We defined a full operational protocol to implement a standard procedure for validation across contributing sources in Europe. The manual described inclusion/exclusion criteria (aged 40+, diagnosis of T2 diabetes at 30+, no prior CVD), risk factors measured over a baseline interval (1/2013-62015), target/competing events at follow-up (2015-2019). We specified a common data model with 9 steps required to process longitudinal records and deliver summary cohort data (one record per person). All rules were implemented in R and NeuBIRO, an original tool written in Java/Groovy and H2 SQL (https://github.com/eubirodnetwork/neubiro). Results Software was able to produce the following outputs at each source: distribution of risk factors by sex and 5-year age groups; Harrell's C-index and standard error of SCORE2 and SCORE2-DM by sex and 10-year age groups, C-index differences; average 5-year predictive vs observed risk by risk deciles; adjusted cumulative incidence of 5-year competing risk by sex and 5-year age groups. Code was packaged into a stand-alone bundle, with test data and coefficients ofthe SCORE2-DM model. The procedures allowed either creating cohort data autonomously to run the supplied R code, or let NeuBIRO complete all steps foreseen to deliver. Tests have been successfully completed in the derivation data, with results from federated databases expected to contribute to the final external validation of SCORE2-DM by midyear. Conclusions We defined a collaborative method to validate risk prediction models in high risk subgroups using international pooling of cohort data with privacy protection. Key messages • Collaborative methods to validate risk prediction models can enhance access to real world data for researchers across chronic diseases. • Implementation of flexible and reusable source code can increase opportunities to use prediction models for robust clinical decision making in multimorbidity.

External validation and updating of prediction models for estimating the 1-year risk of low health-related quality of life in colorectal cancer survivors

ObjectivesTimely identification of colorectal cancer (CRC) survivors at risk of experiencing low health-related quality of life (HRQoL) in the near future is important for enabling appropriately tailored preventive actions. We previously developed and internally validated risk prediction models to estimate the 1-year risk of low HRQoL in long-term CRC survivors. In this article, we aim to externally validate and update these models in a population of short-term CRC survivors. Study Design and SettingIn a pooled cohort of 1,596 CRC survivors, seven HRQoL domains (global QoL, cognitive/emotional/physical/role/social functioning, and fatigue) were measured prospectively at approximately 5 months postdiagnosis (baseline for prediction) and approximately 1 year later by a validated patient-reported outcome measure (European Organization for Research and Treatment of Cancer Quality of life Questionnaire–Core 30). For each HRQoL domain, 1-year scores were dichotomized into low vs. normal/high HRQoL. Performance of the previously developed multivariable logistic prediction models was evaluated (calibration and discrimination). Models were updated to create a more parsimonious predictor set for all HRQoL domains. ResultsUpdated models showed good calibration and discrimination (AUC ≥0.75), containing a single set of 15 predictors, including nonmodifiable (age, sex, education, time since diagnosis, chemotherapy, radiotherapy, stoma, and comorbidities) and modifiable predictors (body mass index, physical activity, smoking, anxiety/depression, and baseline fatigue and HRQoL domain scores). ConclusionExternally validated and updated prediction models performed well for estimating the 1-year risk of low HRQoL in CRC survivors within 6 months postdiagnosis. The impact of implementing the models in oncology practice to improve HRQoL outcomes in CRC survivors needs to be evaluated.

Development and validation of risk prediction models for stroke and mortality among patients with type 2 diabetes in northern China.

Stroke is one of the leading causes of disability and mortality in patients with type 2 diabetes mellitus (T2DM). Risk models have been developed for predicting stroke and stroke-associated mortality among patients with T2DM. Here, we evaluated risk factors of stroke for individualized prevention measures in patients with T2DM in northern China. In the community-based Tianjin Chronic Disease Cohort study, 58,042 patients were enrolled between January 2014 and December 2019. We used multiple imputation (MI) to impute missing variables and univariate and multivariate Cox's proportional hazard regression to screen risk factors of stroke. Furthermore, we established and validated first-ever prediction models for stroke (Model 1 and Model 2) and death from stroke (Model 3) and evaluated their performance. In the derivation and validation groups, the area under the curves (AUCs) of Models 1-3 was better at 5years than at 8years. The Harrell's C-index for all models was above 0.7. All models had good calibration, discrimination, and clinical net benefit. Sensitivity analysis using the MI dataset indicated that all models had good and stable prediction performance. In this study, we developed and validated first-ever risk prediction models for stroke and death from stroke in patients with T2DM, with good discrimination and calibration observed in all models. Based on lifestyle, demographic characteristics, and laboratory examination, these models could provide multidimensional management and individualized risk assessment. However, the models developed here may only be applicable to Han Chinese.

Accommodating population differences when validating risk prediction models.

Validation of risk prediction models in independent data provides a more rigorous assessment of model performance than internal assessment, for example, done by cross-validation in the data used for model development. However, several differences between the populations that gave rise to the training and the validation data can lead to seemingly poor performance of a risk model. In this paper we formalize the notions of "similarity" or "relatedness" of the training and validation data, and define reproducibility and transportability. We address the impact of different distributions of model predictors and differences in verifying the disease status or outcome on measures of calibration, accuracy and discrimination of a model. When individual level information from both the training and validation data sets is available, we propose and study weighted versions of the validation metrics that adjust for differences in the risk factor distributions and in outcome verification between the training and validation data to provide a more comprehensive assessment of model performance. We provide conditions on the risk model and the populations that gave rise to the training and validation data that ensure a model's reproducibility or transportability, and show how to check these conditions using weighted and unweighted performance measures. We illustrate the method by developing and validating a model that predicts the risk of developing prostate cancer using data from two large prostate cancer screening trials.

Abnormal metaphase cytogenetics predicts venous thromboembolism in myeloma: derivation and validation of the PRISM score

AbstractAlthough venous thromboembolism (VTE) is an important treatment and disease-related complication in myeloma, a validated risk prediction model including disease-specific variables such as cytogenetics or tumor burden is lacking. The aim of this study was to develop a new risk prediction model for VTE in the context of modern antimyeloma therapy. All consecutive patients diagnosed at the Cleveland Clinic between 2008 and 2018 and with available data on baseline candidate risk factors constituted the derivation cohort. The primary outcome was VTE (deep venous thrombosis/pulmonary embolism) within 1 year of treatment initiation. A multivariable model was used, and weights were derived from subdistribution hazard ratios to construct a risk score. The model was validated both by internal bootstrap validation and in an external validation cohort. The derivation cohort consisted of 783 patients. A 5-component risk prediction tool, named the PRISM score, was developed, including the following variables: prior VTE, prior surgery, immunomodulatory drug use, abnormal metaphase cytogenetics, and Black race. The c-statistic of the model was 0.622 (95% confidence interval [CI], 0.567-0.674). The model stratified patients into low, intermediate, and high risk, with 12-month cumulative VTE incidence of 2.7%, 10.8%, and 36.5%, respectively. Risk of VTE increased significantly with increasing score in both the derivation and the external validation data sets, with a subdistribution hazard ratio per 1-point increase of 1.28 (95% CI, 1.19-1.39; P < .001) and 1.23 (95% CI, 1.07-1.41; P = .004) respectively. Although the PRISM score can guide clinicians in identifying patients at a high risk of VTE, additional external validation is necessary for incorporation into routine clinical practice.

Usability and Acceptability of Clinical Decision Support Based on the KIIDS-TBI Tool for Children with Mild Traumatic Brain Injuries and Intracranial Injuries.

The Kids Intracranial Injury Decision Support tool for Traumatic Brain Injury (KIIDS-TBI) tool is a validated risk prediction model for managing children with mild traumatic brain injuries (mTBI) and intracranial injuries. Electronic clinical decision support (CDS) may facilitate the clinical implementation of this evidence-based guidance. Our objective was to evaluate the acceptability and usability of an electronic CDS tool for managing children with mTBI and intracranial injuries. Emergency medicine and neurosurgery physicians (10 each) from 10 hospitals in the United States were recruited to participate in usability testing of a novel CDS prototype in a simulated electronic health record environment. Testing included a think-aloud protocol, an acceptability and usability survey, and a semi-structured interview. The prototype was updated twice during testing to reflect user feedback. Usability problems recorded in the videos were categorized using content analysis. Interview transcripts were analyzed using thematic analysis. Among the 20 participants, most worked at teaching hospitals (80%), freestanding children's hospitals (95%), and level-1 trauma centers (75%). During the two prototype updates, problems with clarity of terminology and navigating through the CDS interface were identified and corrected. Corresponding to these changes, the number of usability problems decreased from 35 in phase 1 to 8 in phase 3 and the number of mistakes made decreased from 18 (phase 1) to 2 (phase 3). Through the survey, participants found the tool easy to use (90%), useful for determining a patient's level of care (95%), and likely to improve resource use (90%) and patient safety (79%). Interview themes related to the CDS's ability to support evidence-based decision-making and improve clinical workflow proposed implementation strategies and potential pitfalls. After iterative evaluation and refinement, the KIIDS-TBI CDS tool was found to be highly usable and useful for aiding the management of children with mTBI and intracranial injuries.

Cardiovascular Severe Maternal Morbidity in Pregnant and Postpartum Women: Development and Internal Validation of Risk Prediction Models

(BJOG. 2021;128:922–932. doi: 10.1111/1471-0528.16512. Epub November 4, 2020) Cardiovascular diseases are a leading cause of maternal mortality. Severe maternal morbidity can drastically reduce quality of life, increase hospital lengths of stay, adversely impact finances, and lead to maternal mortality. Thus, it is critical to identify women at high risk for cardiovascular severe maternal morbidity (CSMM) during pregnancy and postpartum and develop strategies to mitigate its adverse effects. This study aims to create and validate risk prediction models so high-risk women can be more accurately identified.

External Validation of Risk Prediction Models to Improve Selection of Patients for Carotid Endarterectomy

Introduction: The net benefit of carotid endarterectomy (CEA) is determined partly by the risk of procedural stroke or death. Current guidelines recommend CEA if the 30 day risks are < 6% for symptomatic stenosis and < 3% for asymptomatic stenosis. We aimed to identify prediction models for procedural stroke or death after CEA, and to externally validate these models in a large registry of patients from the USA.

Reporting Quality of Studies Developing and Validating Melanoma Prediction Models: An Assessment Based on the TRIPOD Statement.

Transparent and accurate reporting is essential to evaluate the validity and applicability of risk prediction models. Our aim was to evaluate the reporting quality of studies developing and validating risk prediction models for melanoma according to the TRIPOD (Transparent Reporting of a multivariate prediction model for Individual Prognosis Or Diagnosis) checklist. We included studies that were identified by a recent systematic review and updated the literature search to ensure that our TRIPOD rating included all relevant studies. Six reviewers assessed compliance with all 37 TRIPOD components for each study using the published “TRIPOD Adherence Assessment Form”. We further examined a potential temporal effect of the reporting quality. Altogether 42 studies were assessed including 35 studies reporting the development of a prediction model and seven studies reporting both development and validation. The median adherence to TRIPOD was 57% (range 29% to 78%). Study components that were least likely to be fully reported were related to model specification, title and abstract. Although the reporting quality has slightly increased over the past 35 years, there is still much room for improvement. Adherence to reporting guidelines such as TRIPOD in the publication of study results must be adopted as a matter of course to achieve a sufficient level of reporting quality necessary to foster the use of the prediction models in applications.

Prediction of Cardiovascular and All-Cause Mortality After Myocardial Infarction in US Veterans

Risk prediction models for cardiovascular disease (CVD) death developed from patients without vascular disease may not be suitable for myocardial infarction (MI) survivors. Prediction of mortality risk after MI may help to guide secondary prevention. Using national electronic record data from the Veterans Health Administration 2002 to 2012, we developed risk prediction models for CVD death and all-cause death based on 5-year follow-up data of 100,601 survivors of MI using Cox proportional hazards models. Model performance was evaluated using a cross-validation approach. During follow-up, there were 31,622 deaths and 12,901 CVD deaths. In men, older age, current smoking, atrial fibrillation, heart failure, peripheral artery disease, and lower body mass index were associated with greater risk of death from CVD or all-causes, and statin treatment, hypertension medication, estimated glomerular filtration rate level, and high body mass index were significantly associated with reduced risk of fatal outcomes. Similar associations and slightly different predictors were observed in women. The estimated Harrell's C-statistics of the final model versus the cross-validation estimates were 0.77 versus 0.77 in men and 0.81 versus 0.77 in women for CVD death. Similarly, the C-statistics were 0.75 versus 0.75 in men, 0.78 versus 0.75 in women for all-cause mortality. The predicted risk of death was well calibrated compared with the observed risk. In conclusion, we developed and internally validated risk prediction models of 5-year risk for CVD and all-cause death for outpatient survivors of MI. Traditional risk factors, co-morbidities, and lack of blood pressure or lipid treatment were all associated with greater risk of CVD and all-cause mortality.

Personalising the risk of conversion from laparoscopic to open hysterectomy in benign conditions: Development and external validation of risk prediction models.

To develop and validate novel prediction models to personalise the risk of conversion from laparoscopic to open hysterectomy in benign conditions. Retrospective cohort study. English NHS Hospitals between 2011 and 2018. 68752 women undergoing laparoscopic hysterectomy for benign conditions. We developed two multivariable logistic models using readily available clinical information, one for the pre-operative setting and another for operative decision-making using additional surgical information, using 2011-2016 data in five regions (24806 women). We validated them (a) temporally in the same regions using 2017-2018 data (12438 women); (b) geographically in the same time-period using data from three different regions (22024 women); and (c) temporally and geographically using 2017-2018 data in three different regions (9484 women). Conversion from laparoscopic to open hysterectomy. Conversions occurred in 6.8% (1687/24806) of hysterectomies in the development group, and in 5.5% (681/12438) in the temporal, 5.9% (1297/22204) in the geographical and 5.2% (488/9484) in the temporal and geographical validation groups. In the development cohort, the area under the receiver operating characteristic curve values for the pre-operative and operative models were 0.65 and 0.67, respectively. In the validation cohorts the corresponding values were 0.65 and 0.66 (temporal), 0.66 and 0.68 (geographical) and 0.65 and 0.68 (temporal and geographical), respectively. Factors predictive of conversion included age, Asian ethnicity, obesity, fibroids, adenomyosis, endometriosis and adhesions. Adhesions were the most predictive (odds ratio 2.54, 95% confidence interval 2.22-2.90). The models for predicting conversions showed acceptable performance and transferability. Novel tool to personalise the risk of conversion from laparoscopic to open hysterectomy in benign conditions.

Comparing LASSO and random forest models for predicting neurological dysfunction among fluoroquinolone users.

Fluoroquinolones are associated with central (CNS) and peripheral (PNS) nervous system symptoms, and predicting the risk of these outcomes may have important clinical implications. Both LASSO and random forest are appealing modeling methods, yet it is not clear which method performs better for clinical risk prediction. To compare models developed using LASSO versus random forest for predicting neurological dysfunction among fluoroquinolone users. We developed and validated risk prediction models using claims data from a commercially insured population. The study cohort included adults dispensed an oral fluoroquinolone, and outcomes were CNS and PNS dysfunction. Model predictors included demographic variables, comorbidities and medications known to be associated with neurological symptoms, and several healthcare utilization predictors. We assessed the accuracy and calibration of these models using measures including AUC, calibration curves, and Brier scores. The underlying cohort contained 16 533 (1.18%) individuals with CNS dysfunction and 46 995 (3.34%) individuals with PNS dysfunction during 120 days of follow-up. For CNS dysfunction, LASSO had an AUC of 0.81 (95% CI: 0.80, 0.82), while random forest had an AUC of 0.80 (95% CI: 0.80, 0.81). For PNS dysfunction, LASSO had an AUC of 0.75 (95% CI: 0.74, 0.76) versus an AUC of 0.73 (95% CI: 0.73, 0.74) for random forest. Both LASSO models had better calibration, with Brier scores 0.17 (LASSO) versus 0.20 (random forest) for CNS dysfunction and 0.20 (LASSO) versus 0.25 (random forest) for PNS dysfunction. LASSO outperformed random forest in predicting CNS and PNS dysfunction among fluoroquinolone users, and should be considered for modeling when the cohort is modest in size, when the number of model predictors is modest, and when predictors are primarily binary.