Risk Stratification Model For Prediction Research Articles

-A machine learning model to predict surgical site infection after surgery of lower extremity fractures.

This study aimed to develop machine learning algorithms for identifying predictive factors associated with the risk of postoperative surgical site infection in patients with lower extremity fractures. A machine learning analysis was conducted on a dataset comprising 1,579 patients who underwent surgical fixation for lower extremity fractures to create a predictive model for risk stratification of postoperative surgical site infection. We evaluated different clinical and demographic variables to train four machine learning models (neural networks, boosted generalised linear model, naïve bayes, and penalised discriminant analysis). Performance was measured by the area under the curve score, Youdon's index and Brier score. A multivariate adaptive regression splines (MARS) was used to optimise predictor selection. The final model consisted of five predictors. (1) Operating room time, (2) ankle region, (3) open injury, (4) body mass index, and (5) age. The best-performing machine learning algorithm demonstrated a promising predictive performance, with an area under the ROC curve, Youdon's index, and Brier score of 77.8%, 62.5%, and 5.1%-5.6%, respectively. The proposed predictive model not only assists surgeons in determining high-risk factors for surgical site infections but also empowers patients to closely monitor these factors and take proactive measures to prevent complications. Furthermore, by considering the identified predictors, this model can serve as a reference for implementing preventive measures and reducing postoperative complications, ultimately enhancing patient outcomes. However, further investigations involving larger datasets and external validations are required to confirm the reliability and applicability of our model.

Development of a predictive model for risk stratification of acute kidney injury in patients undergoing cytoreductive surgery with hyperthermic intraperitoneal chemotherapy

Acute kidney injury (AKI) following hyperthermic intraperitoneal chemotherapy (HIPEC) is common. Identifying patients at risk could have implications for surgical and anesthetic management. We aimed to develop a predictive model that could predict AKI based on patients’ preoperative characteristics and intraperitoneal chemotherapy regimen. We retrospectively gathered data of adult patients undergoing HIPEC at our health system between November 2013 and April 2022. Next, we developed a model predicting postoperative AKI using multivariable logistic regression and calculated the performance of the model (area under the receiver operating characteristics curve [AUC]) via tenfold cross-validation. A total of 412 patients were included, of which 36 (8.7%) developed postoperative AKI. Based on our multivariable logistic regression model, multiple preoperative and intraoperative characteristics were associated with AKI. We included the total intraoperative cisplatin dose, body mass index, male sex, and preoperative hemoglobin level in the final model. The mean area under the receiver operating characteristics curve value was 0.82 (95% confidence interval 0.71–0.93). Our risk model predicted AKI with high accuracy in patients undergoing HIPEC in our institution. The external validity of our model should now be tested in independent and prospective patient cohorts.

Metabolism-Related Prognostic Biomarkers, Purine Metabolism and Anti-Tumor Immunity in Colon Adenocarcinoma.

Metabolic reprogramming provides a new perspective for understanding cancer. The targeting of dysregulated metabolic pathways may help to reprogram the immune status of the tumor microenvironment (TME), thereby increasing the effectiveness of immune checkpoint therapy. Colorectal cancer (CRC), especially colon adenocarcinoma (COAD), is associated with poor patient survival. The aim of the present study was to identify novel pathways involved in the development and prognosis of COAD, and to explore whether these pathways could be used as targets to improve the efficacy of immunotherapy. Metabolism-related differentially expressed genes (MRDEGs) between tumor and normal tissues were identified using The Cancer Genome Atlas (TCGA) dataset, together with metabolism-related prognostic genes (MRPGs). Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway enrichment analysis was performed separately for the MRDEGs and MRPGs. Gene Set Variation Analysis (GSVA) was also performed to explore the role of purine metabolism in COAD tumorigenesis. Consensus clustering of purine metabolism genes with the overall survival (OS) of patients and with anti-tumor immunity was also performed. Pearson correlation analysis was used to identify potential targets that correlated strongly with the expression of immune checkpoints. A 6-gene signature that had independent prognostic significance for COAD was identified, together with a predictive model for risk stratification and prognosis. The most significantly enriched pathway amongst MRDEGs and MRPGs was purine metabolism. Differentially expressed purine metabolism genes could divide patients into two clusters with distinct prognosis and anti-tumor immunity. Further analysis suggested that purine metabolism was involved in anti-tumor immunity. This study confirmed the importance of metabolism-related pathways and in particular purine metabolism in the tumorigenesis, prognosis and anti-tumor immunity of COAD. We identified a 6-gene prognostic signature comprised of EPHX2, GPX3, PTGDS, NAT2, ACOX1 and CPT2. In addition, four potential immune-metabolic checkpoints (GUCY1A1, GUCY1B1, PDE1A and PDE5A) were identified, which could be used to improve the efficacy of immunotherapy in COAD.

Interstitial Brachytherapy for Hepatocellular Carcinoma: Analysis of Prognostic Factors for Overall Survival and Progression-Free Survival and Application of a Risk Stratification Model

Introduction: Interstitial brachytherapy (iBT) is an effective treatment for hepatocellular carcinoma (HCC). Identification of prognostic factors is pivotal for patient selection and treatment efficacy. This study aimed to assess the impact of low skeletal muscle mass (LSMM) on overall survival (OS) and progression-free survival (PFS) of iBT in patients with HCC. Methods: For this single-center study, we retrospectively identified 77 patients with HCC who underwent iBT between 2011 and 2018. Follow-up visits were recorded until 2020. The psoas muscle area, psoas muscle index, psoas muscle density (MD), and the skeletal muscle gauge were assessed on the L3 level on pre-treatment cross-sectional CT scans. Results: Median OS was 37 months. 42 patients (54.5%) had LSMM. An AFP level of >400 ng/ml (hazard ratio [HR] 5.705, 95% confidence interval [CI]: 2.228–14.606, p = 0.001), BCLC stage (HR 3.230, 95% CI: 0.972–10.735, p = 0.026), and LSMM (HR 3.365, 95% CI: 1.490–7.596, p = 0.002) showed a relevant association with OS. Weighted hazard ratios were used to form a predictive risk stratification model with three groups: patients with low risk (median OS 62 months), intermediate risk (median OS 31 months), and high risk (median OS 9 months). The model showed a good prediction of 1-year mortality, with an AUC of 0.71. Higher MD was associated with better PFS (HR 0.920, 95% CI: 0.881–0.962, p < 0.001). Conclusion: In patients undergoing iBT for HCC, LSMM is associated with worse OS. A risk stratification model based on LSMM, AFP >400 ng/mL, and BCLC stage successfully predicted patient mortality. The model may support and enhance patient selection.

Risk stratification using growth differentiation factor 15 in patients undergoing transcatheter aortic valve implantation.

Growth differentiation factor 15 (GDF15) shows potential predictive value in various cardiac conditions. We investigated relationships between GDF15 and clinical or procedural outcomes in patients with severe aortic stenosis undergoing transcatheter aortic valve implantation (TAVI) in order to propose clinically useful predictive risk stratification model. This prospective single-center registry enrolled 88 consecutive patients with severe symptomatic aortic stenosis treated with TAVI. Clinical parameters were collected and biomarkers including GDF-15 were measured within 24 h before TAVI. All relevant clinical outcomes according to the Valve Academic Research Consortium-2 were collected over the follow-up period. The cohort included 52.3% of females. The mean age of study participants was 81 years; the mean Society of Thoracic Surgeons (STS) score and logistic EuroSCORE were 3.6% and 15.4%, respectively. The mortality over the entire follow-up period was 10.2%; no death was observed within the first 30 days following TAVI. Univariate analysis showed significant associations between GDF15 and mortality (P=0.0006), bleeding (P=0.0416) and acute kidney injury (P=0.0399). A standard multivariate logistic regression model showed GDF-15 as the only significant predictor of mortality (P=0.003); the odds ratio corresponding to an increase in GDF15 of 1000 pg/mL was 1.22. However, incremental predictive value was not observed when the STS score was combined with GDF15 in this predictive model. Based on our observations, preprocedural elevated GDF15 levels are associated with increased mortality and demonstrate their additional value in predicting adverse clinical outcomes in a TAVI population.

Development and Validation of a Two-Step Predictive Risk Stratification Model for Coronavirus Disease 2019 In-hospital Mortality: A Multicenter Retrospective Cohort Study.

ObjectivesAn accurate prognostic score to predict mortality for adults with COVID-19 infection is needed to understand who would benefit most from hospitalizations and more intensive support and care. We aimed to develop and validate a two-step score system for patient triage, and to identify patients at a relatively low level of mortality risk using easy-to-collect individual information.DesignMulticenter retrospective observational cohort study.SettingFour health centers from Virginia Commonwealth University, Georgetown University, the University of Florida, and the University of California, Los Angeles.PatientsCoronavirus Disease 2019-confirmed and hospitalized adult patients.Measurements and Main ResultsWe included 1,673 participants from Virginia Commonwealth University (VCU) as the derivation cohort. Risk factors for in-hospital death were identified using a multivariable logistic model with variable selection procedures after repeated missing data imputation. A two-step risk score was developed to identify patients at lower, moderate, and higher mortality risk. The first step selected increasing age, more than one pre-existing comorbidities, heart rate >100 beats/min, respiratory rate ≥30 breaths/min, and SpO2 <93% into the predictive model. Besides age and SpO2, the second step used blood urea nitrogen, absolute neutrophil count, C-reactive protein, platelet count, and neutrophil-to-lymphocyte ratio as predictors. C-statistics reflected very good discrimination with internal validation at VCU (0.83, 95% CI 0.79–0.88) and external validation at the other three health systems (range, 0.79–0.85). A one-step model was also derived for comparison. Overall, the two-step risk score had better performance than the one-step score.ConclusionsThe two-step scoring system used widely available, point-of-care data for triage of COVID-19 patients and is a potentially time- and cost-saving tool in practice.

A new prediction model integrated serum lipid profile for patients with multiple myeloma.

Purpose: This study aimed to explore a predictive risk-stratification model combing clinical characteristics and lipid profiles in multiple myeloma (MM) patients.Methods: The data of 275 patients in Sun Yat-Sen University Cancer Center were retrospectively analyzed and randomly divided into the training (n = 138) and validation (n=137) cohorts. Triglyceride (TG), low-density lipoprotein (LDL), high-density lipoprotein (HDL), lactate dehydrogenase (LDH), Apolipoprotein B (Apo B) and Apo B/Apolipoprotein A1 (Apo A1) ratio were the prognostic factors identified through univariate and multivariate Cox analysis.Results: A 6-prognostic factor model was constructed based on Lasso regression. Patients were divided into low- and high-risk groups and the former group showed longer overall survival (OS) time (p<0.05). The area under the curve (AUC) of the risk score model for 5-and 10-year OS were 0.756 [95% CI: 0.661-0.850] and 0.940 [95% CI: 0.883-0.997], which exhibited better accuracy than International Staging System (ISS) and Durie and Salmon (DS) stage.Conclusion: This study aims to combine the lipid metabolism profile with the clinical characteristics of MM patients to generate a prognostic model. The nomogram integrating ISS stage and risk score increased the prediction accuracy. This model can monitor lipid profile as a simple and effective method, which has certain clinical significance for improving the accuracy of the prognosis and exploring potential therapeutic targets.

A clinical predictive model for risk stratification of patients with severe acute lower gastrointestinal bleeding

BackgroundLower gastrointestinal bleeding (LGIB) is a common presentation of surgical admissions, imposing a significant burden on healthcare costs and resources. There is a paucity of standardised clinical predictive tools available for the initial assessment and risk stratification of patients with LGIB. We propose a simple clinical scoring model to prognosticate patients at risk of severe LGIB and an algorithm to guide management of such patients.MethodsA retrospective cohort study was conducted, identifying consecutive patients admitted to our institution for LGIB over a 1-year period. Baseline demographics, clinical parameters at initial presentation and treatment interventions were recorded. Multivariate logistic regression was performed to identify factors predictive of severe LGIB. A clinical management algorithm was developed to discriminate between patients requiring admission, and to guide endoscopic, angiographic and/or surgical intervention.Results226/649 (34.8%) patients had severe LGIB. Six variables were entered into a clinical predictive model for risk stratification of LGIB: Tachycardia (HR ≥ 100), hypotension (SBP < 90 mmHg), anaemia (Hb < 9 g/dL), metabolic acidosis, use of antiplatelet/anticoagulants, and active per-rectal bleeding. The optimum cut-off score of ≥ 1 had a sensitivity of 91.9%, specificity of 39.8%, and positive and negative predictive Values of 45% and 90.2%, respectively, for predicting severe LGIB. The area under curve (AUC) was 0.77.ConclusionEarly diagnosis and management of severe LGIB remains a challenge for the acute care surgeon. The predictive model described comprises objective clinical parameters routinely obtained at initial triage to guide risk stratification, disposition and inpatient management of patients.

Implementing emergency admission risk prediction in general practice: a qualitative study.

BackgroundUsing computer software in general practice to predict patient risk of emergency hospital admission has been widely advocated, despite limited evidence about effects. In a trial evaluating the introduction of a Predictive Risk Stratification Model (PRISM), statistically significant increases in emergency hospital admissions and use of other NHS services were reported without evidence of benefits to patients or the NHS.AimTo explore GPs’ and practice managers’ experiences of incorporating PRISM into routine practice.Design and settingSemi-structured interviews were carried out with GPs and practice managers in 18 practices in rural, urban, and suburban areas of south Wales.MethodInterviews (30–90 min) were conducted at 3–6 months after gaining PRISM access, and ∼18 months later. Data were analysed thematically using Normalisation Process Theory.ResultsResponders (n = 22) reported that the decision to use PRISM was based mainly on fulfilling Quality and Outcomes Framework incentives. Most applied it to <0.5% practice patients over a few weeks. Using PRISM entailed undertaking technical tasks, sharing information in practice meetings, and making small-scale changes to patient care. Use was inhibited by the model not being integrated with practice systems. Most participants doubted any large-scale impact, but did cite examples of the impact on individual patient care and reported increased awareness of patients at high risk of emergency admission to hospital.ConclusionQualitative results suggest mixed views of predictive risk stratification in general practice and raised awareness of highest-risk patients potentially affecting rates of unplanned hospital attendance and admissions. To inform future policy, decision makers need more information about implementation and effects of emergency admission risk stratification tools in primary and community settings.

Machine Learning-based Model for Predicting Postoperative Complications among Patients with Colonic Perforation: A Retrospective study

Objectives: Surgery for colonic perforation has high morbidity and mortality rates. Predicting complications preoperatively would help improve short-term outcomes; however, no predictive risk stratification model exists to date. Therefore, the current study aimed to determine risk factors for complications after colonic perforation surgery and use machine learning to construct a predictive model. Methods: This retrospective study included 51 patients who underwent emergency surgery for colorectal perforation. We investigated the connection between overall complications and several preoperative indicators, such as lactate and the Glasgow Prognostic Score. Moreover, we used the classification and regression tree (CART), a machine-learning method, to establish an optimal prediction model for complications. Results: Overall complications occurred in 32 patients (62.7%). Multivariate logistic regression analysis identified high lactate levels [odds ratio (OR), 1.86; 95% confidence interval (CI), 1.07-3.22; p = 0.027] and hypoalbuminemia (OR, 2.56; 95% CI, 1.06-6.25; p = 0.036) as predictors of overall complications. According to the CART analysis, the albumin level was the most important parameter, followed by the lactate level. This prediction model had an area under the curve (AUC) of 0.830. Conclusions: Our results determined that both preoperative albumin and lactate levels were valuable predictors of postoperative complications among patients who underwent colonic perforation surgery. The CART analysis determined optimal cutoff levels with high AUC values to predict complications, making both indicators clinically easier to use for decision making.

Machine learning and statistical methods for predicting mortality in heart failure.

Heart failure is a debilitating clinical syndrome associated with increased morbidity, mortality, and frequent hospitalization, leading to increased healthcare budget utilization. Despite the exponential growth in the introduction of pharmacological agents and medical devices that improve survival, many heart failure patients, particularly those with a left ventricular ejection fraction less than 40%, still experience persistent clinical symptoms that lead to an overall decreased quality of life. Clinical risk prediction is one of the strategies that has been implemented for the selection of high-risk patients and for guiding therapy. However, most risk predictive models have not been well-integrated into the clinical setting. This is partly due to inherent limitations, such as creating risk predicting models using static clinical data that does not consider the dynamic nature of heart failure. Another limiting factor preventing clinicians from utilizing risk prediction models is the lack of insight into how predictive models are built. This review article focuses on describing how predictive models for risk-stratification of patients with heart failure are built.

Homegrown Safety Net System Predictive Model for Risk Stratification: Assessing Potential Racial Bias

Research ObjectiveIncreasingly leveraged in health care, artificial intelligence and machine learning techniques are drawing concerns about biases and racial disparities. Proprietary “black box” predictive models offer limited understanding of algorithmic bias and potential impact for patients. From 2016 to 2018, safety‐net health system NYC Health + Hospitals (H+H) developed a payer‐agnostic algorithm predicting future acute care utilization from past utilization, social determinants, demographics, insurance, and number of chronic conditions. High‐risk patients are eligible for augmented care management. We sought to assess potential racial biases in our predictive model of acute care utilization.Study DesignWe examined bias in predicted vs. actual utilization among our model validation cohort. Stratifying by non‐Hispanic White (White), non‐Hispanic Black (Black), Hispanic, and Other race/ethnicity categories, we compared (a) distribution of high‐risk status; (b) model sensitivity and positive predictive value (PPV); and (c) phenotype of “false negative” patients who were not flagged as high risk but became high utilizers. We conducted sensitivity analyses to explore the impact of prediction threshold on bias for top 1% (extreme high risk) vs top 5% (implemented cutoff).Population StudiedValidation cohort patients (N = 250 191) were randomly selected and represented 30% of adults who visited H+H from July 2017 to June 2018. Incarcerated or pregnant patients were excluded.Principal FindingsPredicted distribution of race/ethnicity among high‐risk patients was 44.4% Black, 22.9% Hispanic, 19.4% Other, and 13.3% White. Actual distribution among high utilizing patients was 40.5% Black, 31.1% Hispanic, 19.4% Other, and 9% White. Black and White patients were overrepresented among predicted high‐risk patients, and Hispanic patients were underrepresented.PPVs across race/ethnicity were comparable (&lt;5% absolute difference) but sensitivities varied greatly. The model was best at capturing risk among White patients (sensitivity: 35.5%), followed by Black patients (29.0%), and Other patients (25.0%). Hispanic patients had the lowest sensitivity (18.2%).In phenotyping our false negatives, the model more frequently failed to identify female patients, particularly Hispanic, Other, or Black females; and uninsured patients, particularly Hispanic uninsured patients, suggesting possible interactions with race/ethnicity, gender, and insurance status. Sensitivity among Hispanic patients varied greatly by insurance status, from 27.5% (Medicaid) to 8.9% (uninsured). Differences by race/ethnicity were attenuated using the stricter top 1% definition of high risk in our sensitivity analyses.ConclusionsUnlike cost‐oriented algorithms like that evaluated by Obermeyer et al, we did not find bias against Black patients in H + H’s safety net risk prediction model. Our model did under‐predict risk for Hispanic patients. Understanding the sources of this difference will require further examination. Preliminary studies suggest Hispanic patients may use the system differently, engaging frequently in primary care but also under‐utilizing health care overall due to insurance status. We are now evaluating the value of applying different risk thresholds by subpopulation to mitigate the effects of algorithmic bias for our patients.Implications for Policy or PracticeIn December 2019 following Obermeyer et al’s article on model bias in Science, Sen. Ron Wyden and Sen. Cory Booker urged federal agencies and insurance companies to examine the prevalence and impact of bias in health care algorithms. This study provides an additional framework for those evaluations.

Rationale and design of multicenter prospective observational study of types, GRAde, VariabilITY, associations and prognosis of orthostatic responses in Heart Failure (GRAVITY-HF)

Heart failure (HF) is one of the leading causes of adult mortality. Increased risk of death determines need for better understanding of the pathophysiological mechanisms, predictive risk stratification models and applicable methods to improve prognosis. One of the unfavorable prognostic factors may be an inadequate hemodynamic response to orthostatic stress. Orthostatic hypotension (OH) is known to be an independent predictor of many cardiovascular diseases, particularly HF, and death. Singlecenter study of HF population revealed that systolic blood pressure within 3-5 minutes after standing up may be a predictor of long-term unfavorable outcomes. Nevertheless, data about OH in patients with HF are limited and inconsistent due to heterogeneity of populations and different methodology in published studies. In this regard, a population-based study of the orthostatic response in patients with HF (stable and decompensated) is needed. The article describes the rationale and design of a multicenter prospective observational study aimed to assess the clinical and prognostic significance of orthostatic responses in HF patients.

Identifying unreliable predictions in clinical risk models

The ability to identify patients who are likely to have an adverse outcome is an essential component of good clinical care. Therefore, predictive risk stratification models play an important role in clinical decision making. Determining whether a given predictive model is suitable for clinical use usually involves evaluating the model’s performance on large patient datasets using standard statistical measures of success (e.g., accuracy, discriminatory ability). However, as these metrics correspond to averages over patients who have a range of different characteristics, it is difficult to discern whether an individual prediction on a given patient should be trusted using these measures alone. In this paper, we introduce a new method for identifying patient subgroups where a predictive model is expected to be poor, thereby highlighting when a given prediction is misleading and should not be trusted. The resulting “unreliability score” can be computed for any clinical risk model and is suitable in the setting of large class imbalance, a situation often encountered in healthcare settings. Using data from more than 40,000 patients in the Global Registry of Acute Coronary Events (GRACE), we demonstrate that patients with high unreliability scores form a subgroup in which the predictive model has both decreased accuracy and decreased discriminatory ability.

Risk Stratification Model for Prediction of Locoregional Recurrence in Patients with Pathologic T1-2N0 Breast Cancer after Mastectomy

Risk Stratification Model for Prediction of Locoregional Recurrence in Patients with Pathologic T1-2N0 Breast Cancer after Mastectomy

Effects and costs of implementing predictive risk stratification in primary care: a randomised stepped wedge trial

AimWe evaluated the introduction of a predictive risk stratification model (PRISM) into primary care. Contemporaneously National Health Service (NHS) Wales introduced Quality and Outcomes Framework payments to general practices to...

Risk stratification in deep brain stimulation surgery: Development of an algorithm to predict patient discharge disposition with 91.9% accuracy

Clinical decision making is susceptible to biases and can be improved with the application of predictive models and decision support systems (DSS). The purpose of this study was to develop a predictive risk stratification model and DSS that could accurately predict whether a patient was likely to be of high- or low-acuity discharge disposition (DD) status subsequent to DBS surgery. Data were collected for 135 DBS patients by reviewing medical records. Multivariate logistic regression was applied to develop the predictive algorithm. The two significant predictive models showed good fit and were calibrated by using AUROC curve analysis. The model predicting DD in all patients (n = 135) yielded a predictive accuracy of 91.9% (AUROC = 0.825, p < .001). The model predicting DD in Parkinson’s Disease patients (n = 91) yielded a predictive accuracy of 89.0% (AUROC = 0.853, p < .001). Age was a significant predictor of DD across all patients (OR = 1.11, p < .05) and BMI was a significant predictor of DD amongst Parkinson’s Disease patients (OR = 1.16, p < .05). It is possible to accurately predict the DD of DBS patients using routinely collected preoperative variables. The predictive algorithms were applied in the form of a model-driven DSS to assist in improving clinical decision making and patient safety.

Development of predictive scoring model for risk stratification of no-show at a public hospital specialist outpatient clinic

Aim: No-shows are patients who miss scheduled specialist outpatient clinic (SOC) appointments. A predictive scoring model for the risk stratification of no-shows was developed to improve the utilisation of resources. Method: The administrative records of new SOC appointments for subsidised patients in 2013 were analysed. Univariate analysis was performed on 16 variables comprising patient demographics, appointment/visit records and historical outpatient records. Multiple logistic regression (MLR) was applied to determine independent risk factors of no-shows. The adjusted parameter estimates from MLR were used to develop a predictive model for risk stratification of no-show. Model validation was performed using 2014 data. Result: Out of 75,677 appointments in 2013, 28.6% were no-shows. Univariate analysis showed that 11 variables were associated with no-shows. Six variables (age, race, specialty, lead time, referral source, previous visit status) remained independently associated with no-shows in the MLR model, and their odds ratios were used to develop the weighted predictive scoring model. Weighted scores were 0 to 19, and five levels of no-show risk were derived: extremely low (score: 0–4; odds ratio (OR): 1.0); low (5–6; OR: 2.5); medium (7–8; OR: 5.6); high (9–10; OR: 9.2); and extremely high (11–19; OR: 16.7). The predictive ability of the model was tested using receiver operation curve analysis, where the area under curve (AUC) was 72%. AUC remained at 72% upon validation with 2014 data. Conclusion: The prediction model developed using only administrative data was robust and can be used for the risk stratification of SOC no-show for better resource utilisation to improve access to care.

Predictive risk stratification model: a randomised stepped-wedge trial in primary care (PRISMATIC)

BackgroundWith a higher proportion of older people in the UK population, new approaches are needed to reduce emergency hospital admissions, thereby shifting care delivery out of hospital when possible and safe.Study aimTo evaluate the introduction of predictive risk stratification in primary care.ObjectivesTo (1) measure the effects on service usage, particularly emergency admissions to hospital; (2) assess the effects of the Predictive RIsk Stratification Model (PRISM) on quality of life and satisfaction; (3) assess the technical performance of PRISM; (4) estimate the costs of PRISM implementation and its effects; and (5) describe the processes of change associated with PRISM.DesignRandomised stepped-wedge trial with economic and qualitative components.SettingAbertawe Bro Morgannwg University Health Board, south Wales.ParticipantsPatients registered with 32 participating general practices.InterventionPRISM software, which stratifies patients into four (emergency admission) risk groups; practice-based training; and clinical support.Main outcome measuresPrimary outcome – emergency hospital admissions. Secondary outcomes – emergency department (ED) and outpatient attendances, general practitioner (GP) activity, time in hospital, quality of life, satisfaction and costs.Data sourcesRoutine anonymised linked health service use data, self-completed questionnaires and staff focus groups and interviews.ResultsAcross 230,099 participants, PRISM implementation led to increased emergency admissions to hospital [ΔL = 0.011, 95% confidence interval (CI) 0.010 to 0.013], ED attendances (ΔL = 0.030, 95% CI 0.028 to 0.032), GP event-days (ΔL = 0.011, 95% CI 0.007 to 0.014), outpatient visits (ΔL = 0.055, 95% CI 0.051 to 0.058) and time spent in hospital (ΔL = 0.029, 95% CI 0.026 to 0.031). Quality-of-life scores related to mental health were similar between phases (Δ = –0.720, 95% CI –1.469 to 0.030); physical health scores improved in the intervention phase (Δ = 1.465, 95% CI 0.774 to 2.157); and satisfaction levels were lower (Δ = –0.074, 95% CI – 0.133 to –0.015). PRISM implementation cost £0.12 per patient per year and costs of health-care use per patient were higher in the intervention phase (Δ = £76, 95% CI £46 to £106). There was no evidence of any significant difference in deaths between phases (9.58 per 1000 patients per year in the control phase and 9.25 per 1000 patients per year in the intervention phase). PRISM showed good general technical performance, comparable with existing risk prediction tools (c-statistic of 0.749). Qualitative data showed low use by GPs and practice staff, although they all reported using PRISM to generate lists of patients to target for prioritised care to meet Quality and Outcomes Framework (QOF) targets.LimitationsIn Wales during the study period, QOF targets were introduced into general practice to encourage targeting care to those at highest risk of emergency admission to hospital. Within this dynamic context, we therefore evaluated the combined effects of PRISM and this contemporaneous policy initiative.ConclusionsIntroduction of PRISM increased emergency episodes, hospitalisation and costs across, and within, risk levels without clear evidence of benefits to patients.Future research(1) Evaluation of targeting of different services to different levels of risk; (2) investigation of effects on vulnerable populations and health inequalities; (3) secondary analysis of the Predictive Risk Stratification: A Trial in Chronic Conditions Management data set by health condition type; and (4) acceptability of predictive risk stratification to patients and practitioners.Trial and study registrationCurrent Controlled Trials ISRCTN55538212 and PROSPERO CRD42015016874.FundingThe National Institute for Health Research Health Services Delivery and Research programme.

Invited Commentary

Using data from the United Network for Organ Sharing (UNOS) publicly available, deidentified Scientific Registry of Transplant Recipients (SRTR) database, Crawford and colleagues [1Crawford T.C. Magruder J.T. Grimm J.C. et al.A comprehensive risk score to predict prolonged hospital length of stay after heart transplantation.Ann Thorac Surg. 2018; 105: 83-91Abstract Full Text Full Text PDF PubMed Scopus (14) Google Scholar] performed a multivariate logistic regression analysis to develop a predictive risk score for prolonged postoperative hospital length of stay in a cohort of isolated, adult heart transplantation recipients who survived to discharge from 2003 to 2012. Notable exclusions from their analysis included patients with missing postoperative length of stay data and all patients who died during the postoperative period during their index hospitalization before discharge. Using an approximation of the 90th percentile of length of stays for the entire cohort, the authors defined a prolonged postoperative hospital length of stay as a posttransplant hospital length of stay greater than 30 days. Using the c-statistic to evaluate for discrimination, the authors found their model to be modestly accurate at best (c-statistic 0.61). Multivariate logistic regression has been the cornerstone on which cardiac surgery risk estimation models have been developed over the last 3 decades. The methodology assumes fixed, weighted combinations of independent variables (input) and linearly relates them to the dependent (output) variables under evaluation. However, using logistic regression to model complex postoperative outcomes is insufficient as this method is unable to correctly evaluate the nonlinear and indirect relationships that characterize more complicated clinical scenarios, leading to more often than not inaccurately published risk prediction models. One potential way of more accurately assessing these nonlinear relationships is through the use of advanced machine learning statistical methods. Machine learning techniques, coupled with readily available large electronic datasets and open-source statistical software, can integrate massive feature sets to model many interrelated, nonlinear relationships and higher order interactions. Examples of machine learning methods arriving at more accurate clinical risk prediction models are present throughout the medical literature but are also now beginning to dot the cardiac surgery evidence base as well. Examples include the Bayesian network approach of Loghmanpour and colleagues [2Loghmanpour N.A. Kanwar M.K. Druzdzel M.J. Benza R.L. Murali S. Antaki J.F. A new Bayesian network-based risk stratification model for prediction of short-term and long-term LVAD mortality.ASAIO J. 2015; 61: 313-323Crossref PubMed Scopus (34) Google Scholar] for short- and long-term mortality risk in left ventricular assist device patients using the Interagency Registry for Mechanically Assisted Circulatory Support (INTERMACS) data; and the more recent publication by Allyn and colleagues [3Allyn J. Allou N. Augustin P. et al.A comparison of a machine learning model with EuroSCORE II in predicting mortality after elective cardiac surgery: a decision curve analysis.PLoS One. 2017; 12: e0169772Crossref PubMed Scopus (95) Google Scholar] of an ensemble machine learning risk prediction model that outperformed the European System for Cardiac Operative Risk Evaluation (EuroSCORE) II in a cohort of 6,520 cardiac surgery patients from a single center in France. The idea behind building a risk prediction model with forced linear relationships results in publication of many inaccurate models. Instead, we should strive to assess the true nature of the underlying relationships between clinical-level features through machine learning methodologies in hopes of developing more accurate risk stratification and prediction scores that will help us more precisely inform patient counseling and surgeon practice. A Comprehensive Risk Score to Predict Prolonged Hospital Length of Stay After Heart TransplantationThe Annals of Thoracic SurgeryVol. 105Issue 1PreviewProlonged hospital length of stay (PLOS) after heart transplantation increases cost and morbidity. To better inform care, we developed a risk score to identify patients at risk for PLOS after heart transplantation. Full-Text PDF