Case-mix Adjustment Models Research Articles

331. CASE-MIX ADJUSTMENT IN BETWEEN-HOSPITAL COMPARISONS ON CLINICAL OUTCOMES IN ESOPHAGEAL AND GASTRIC CANCER SURGERY: A NATIONWIDE POPULATION-BASED COHORT STUDY

Abstract Background Clinical outcomes of esophageal and gastric cancer surgery are used for internal and external benchmarking in clinical auditing. For true hospital comparisons, proper case-mix adjustment is required. This study aimed to develop distinct models that allow for case-mix adjusted quality assessment of esophageal and gastric cancer surgery separately and assessed the impact on between-hospital comparisons. Methods This study included all patients undergoing esophagogastric cancer surgery in the Netherlands between 2017-2022 registered in the Dutch Upper Gastrointestinal Cancer Audit. We developed distinct case-mix adjustment models for ten quality indicators (QIs) using backward selection. Model performance of each individual model was evaluated with area-under-the-receiving-operator-curve (AUC) statistics, representing the impact of case-mix on the QI scores. The impact on between-hospital comparisons was quantified using unadjusted and adjusted O/E ratios. Results A total of 4,354 esophageal cancer and 2,109 gastric cancer patients were included. The most frequently selected case-mix variables in the models for all QIs for esophageal cancer surgery were ASA-score, salvage surgery, peripheral vascular disease/aortic aneurysm, chronic lung disease, and tumor histology, whereas for gastric cancer these were age, preoperative weight loss, tumor location, and clinical M stage. The case-mix adjustment models for esophageal and gastric cancer surgery showed low to moderate performance, with internally validated AUCs of 0.58-0.73 and 0.58-0.74, respectively. Case-mix adjustment had low impact on between-hospital comparisons. For both types of cancer, case-mix adjustment had most pronounced impact on between-hospital comparisons for QIs 30-day/in-hospital mortality and failure-to-rescue. Conclusion This study showed that correct quality assessment requires different case-mix models per quality indicator for clinical outcomes of esophageal and gastric cancer surgery. The case-mix models had low to moderate performance scores, suggesting that case-mix has limited impact in between-hospital comparisons of oesopagogastric cancer surgery.

Association of multimorbidity with mortality after stroke stratified by age, severity, etiology, and prior disability.

Multimorbidity is common in patients with stroke and is associated with increased medium- to long-term mortality, but its value for clinical decision-making and case-mix adjustment will depend on other factors, such as age, stroke severity, etiological subtype, prior disability, and vascular risk factors. In the absence of previous studies, we related multimorbidity to long-term post-stroke mortality with stratification by these factors. In patients ascertained in a population-based stroke incidence study (Oxford Vascular Study; 2002-2017), we related pre-stroke multimorbidity (weighted/unweighted Charlson comorbidity index (CCI)) to all-cause/vascular/non-vascular mortality (1/5/10 years) using regression models adjusted/stratified by age, sex, predicted early outcome (THRIVE score), stroke severity (NIH stroke scale (NIHSS)), etiology (Trial of Org 10172 in Acute Stroke Treatment (TOAST)), premorbid disability (modified Rankin Scale (mRS)), and non-CCI risk factors (hypertension, hyperlipidemia, atrial fibrillation, smoking, deprivation, anxiety/depression). Among 2454 stroke patients (M/SD age: 74.1/13.9 years; 48.9% male; M/SD NIHSS: 5.7/7.0), 1375/56.0% had ⩾ 1 CCI comorbidity and 685/27.9% had ⩾ 2. After age/sex adjustment, multimorbidity (unweighted CCI ⩾ 2 vs 0) predicted (all ps < 0.001) mortality at 1 year (aHR = 1.57, 95% CI = 1.38-1.78), 5 years (aHR = 1.73, 95% CI = 1.53-1.96), and 10 years (aHR = 1.79, 95% CI = 1.58-2.03). Although multimorbidity was independently associated with premorbid disability (mRS > 2: aOR = 2.76, 2.13-3.60) and non-CCI risk factors (hypertension: 1.56, 1.25-1.95; hyperlipidemia: 2.58, 2.03-3.28; atrial fibrillation: 2.31; 1.78-2.98; smoking: 1.37, 1.01-1.86), it predicted death after adjustment for all measured confounders (10-year-aHR = 1.56, 1.37-1.78, p < 0.001), driven mainly by non-vascular death (aHR = 1.89, 1.55-2.29). Predictive value for 10-year all-cause death was greatest in patients with lower expected early mortality: lower THRIVE score (pint < 0.001), age < 75 years (aHR = 2.27, 1.71-3.00), NIHSS < 5 (1.84, 1.53-2.21), and lacunar stroke (3.56, 2.14-5.91). Results were similar using the weighted CCI. Pre-stroke multimorbidity is highly prevalent and is an independent predictor of death after stroke, supporting its inclusion in case-mix adjustment models and in informing decision-making by patients, families, and carers. Prediction in younger patients and after minor stroke, particularly for non-vascular death, suggests potential clinical utility in targeting interventions that require survival for 5-10 years to achieve a favorable risk/benefit ratio. Data requests will be considered by the Oxford Vascular Study (OXVASC) Study Director (P.M.R.-peter.rothwell@ndcn.ox.ac.uk).

Temporal Changes in Quality Indicators in a Regional System of Care After Surgical and Transcatheter Aortic Valve Replacement

BackgroundHistorically, quality-of-care monitoring was performed separately for transcatheter and surgical aortic valve replacement (TAVR, SAVR). Using consensus indicators, we provide a global report on the quality of care for treatment of aortic stenosis across the highest-volume treatments: transfemoral (TF) TAVR, isolated SAVR, and SAVR combined with coronary artery bypass graft. MethodsRetrospective observational cohort study of consecutive patients in a regional system of care. Primary endpoint was 30-day and 1-year mortality (2015-2019). Secondary endpoints included rate of new pacemaker, rate of readmission, and length of stay (2012-2019). Following multivariable logistic regressions, we developed mortality case-mix adjustment models to report risk estimates. ResultsThe proportion of patients receiving TAVR grew from 32% to 53% (2015-2019). Those receiving TF TAVR were significantly older, with higher rates of comorbidities. Observed 30-day and 1-year all-cause mortality after TF TAVR decreased from 3.1% to 0.6% (P = 0.03), and 13.6% to 6.6% (P = 0.09), respectively; surgical mortality rates for isolated SAVR and SAVR combined with coronary artery bypass graft were low and did not change significantly over time, ranging from 0.3% to 1.4% and from 0.9% to 3.4%, respectively at 30 days, and from 0.9% to 3.4% and from 4.7% to 6.7 at 1 year. In the TF TAVR cohort, the observed vs expected ratio for 30-day and 1-year mortality decreased significantly from 1.9 (95% confidence interval [CI] 0.9, 3.5) to 0.3 (95% CI 0.1, 0.8), and from 1.3 (95% CI 0.9, 1.7) to 0.7 (95% CI 0.5, 0.99), respectively; no change occurred in risk-adjusted surgical mortality. ConclusionsConsensus quality indicators provide unique insights on the quality of care for patients receiving treatment for aortic stenosis.

Case-mix adjustment of patient-reported experience measures in National Regional Center for Pediatric.

The aim of the study was to identify case-mix adjusters for the Chinese version of the Child Hospital Consumer Assessment of Healthcare Providers and Systems (Child-HCAHPS) and assess the impact of case-mix adjustment on patient experience measures in China. This study analyzed data collected from six National Regional Center for Pediatric across China retrospectively. Participants were children aged ≤17 years and their guardians who completed the survey. The Chinese Child-HCAHPS was used to measure pediatric inpatient care experience. Candidate case-mix adjusters were assessed using a summary measure of explanatory power. Changes in scores and rankings of the six centers were quantified to assess the impact of adjustment. A total of 2708 respondents completed the survey from January to March 2021, with a response rate of 7-15%. The child's global health status and the respondent being the child's mother were identified as case-mix adjusters, and case-mix adjustment models for 18 patient experience items were constructed. Kendall's τ correlation of hospital rankings before and after adjustment ranged from 0.73 to 1.00. Although the impact of case-mix adjustment may appear modest in our sample, it demonstrated the feasibility, necessity, and methodology for further development of case-mix adjustment models in pediatric healthcare facilities in China. Case-mix adjustment models adjust for factors that are unamendable by healthcare providers that may affect patient experience ratings, thereby improving the comparability of institutional-level ratings. Standardized case-mix adjustment protocols for quality measures need to be modified in different settings. This is the first study to identify adjustment variables and the possible impact of case-mix adjustment on pediatric inpatients' experience measures in a Chinese population. This study provided evidence on the feasibility and necessity for further development of case-mix adjustment models for pediatric healthcare facilities in China.

Musculoskeletal case-mix adjustment in a UK primary/community care cohort: Testing musculoskeletal models to make recommendations in this setting

Benchmarking musculoskeletal (MSK) services is limited by the need to adjust for differences in patient characteristics/case-mix. Without this providers and services cannot be usefully compared. This paper investigates the predictive ability of case-mix adjustment models in a primary/community care cohort. ObjectivesTo investigate the predictive ability of two existing MSK case-mix adjustment models and compare to the predictive ability of an evidence informed and statistically informed model. MethodA secondary analysis of the ‘Subgrouping for Targeted Treatment in Musculoskeletal Conditions’ cluster randomised controlled trial data (n = 1211). Stepwise linear regression models were built and compared using available baseline variables. The MSK-HQ was used as the primary functional status outcome. ResultsTwo existing models were compared (UK National PROMs Model, US FOTO Model) using available variables. Of these models the modified US FOTO model showed the best predictive ability in this cohort predicting 44% of the variation in MSK-HQ outcome, the modified UK National PROMs model predicted 41%. A newly developed evidence informed model (Keele Model 1) performed no better than the existing models, and a statistically informed model (Keele Model 2) gave only an additional 2% increase in model power compared to the modified US FOTO model. ConclusionAll models showed strong predictive ability. The modified US FOTO model looks to be best suited to the UK primary/community care cohort of the existing models. This model performed so well that we recommend that this model is used in a UK setting moving forwards rather than development of an alternative UK model.

Value-based healthcare in ischemic stroke care: case-mix adjustment models for clinical and patient-reported outcomes

BackgroundPatient-Reported Outcome Measures (PROMs) have been proposed for benchmarking health care quality across hospitals, which requires extensive case-mix adjustment. The current study’s aim was to develop and compare case-mix models for mortality, a functional outcome, and a patient-reported outcome measure (PROM) in ischemic stroke care.MethodsData from ischemic stroke patients, admitted to four stroke centers in the Netherlands between 2014 and 2016 with available outcome information (N = 1022), was analyzed. Case-mix adjustment models were developed for mortality, modified Rankin Scale (mRS) scores and EQ-5D index scores with respectively binary logistic, proportional odds and linear regression models with stepwise backward selection. Predictive ability of these models was determined with R-squared (R2) and area-under-the-receiver-operating-characteristic-curve (AUC) statistics.ResultsAge, NIHSS score on admission, and heart failure were the only common predictors across all three case-mix adjustment models. Specific predictors for the EQ-5D index score were sex (β = 0.041), socio-economic status (β = − 0.019) and nationality (β = − 0.074). R2-values for the regression models for mortality (5 predictors), mRS score (9 predictors) and EQ-5D utility score (12 predictors), were respectively R2 = 0.44, R2 = 0.42 and R2 = 0.37.ConclusionsThe set of case-mix adjustment variables for the EQ-5D at three months differed considerably from the set for clinical outcomes in stroke care. The case-mix adjustment variables that were specific to this PROM were sex, socio-economic status and nationality. These variables should be considered in future attempts to risk-adjust for PROMs during benchmarking of hospitals.

Profiling Hospital Performance Based on Mortality After Transcatheter Aortic Valve Replacement in Ontario, Canada.

Public reporting of hospital-level outcomes is increasingly common as a means to target quality improvement strategies to ensure the delivery of optimal care. Despite the rapid dissemination of transcatheter aortic valve replacement (TAVR), there is a paucity of reliable case-mix adjustment models for hospital profiling in TAVR. Our objective was to develop and evaluate different models for calculating risk-standardized all-cause mortality rates (RSMRs) post-TAVR. In this population-based study in Ontario, Canada, we identified all patients who underwent a TAVR procedure between April 1, 2012, and March 31, 2016. For each hospital, we calculated 30-day and 1-year RSMR, using 2-level hierarchical logistic regression models that accounted for patient-specific demographic and clinical characteristics, as well as the clustering of patients within the same hospital using a hospital-specific random effects. We classified each hospital into one of 3 groups: performing worse than expected, better than expected, or performing as expected, based on whether the 95% CI of the RSMR was above, below, or included the provincial average mortality rate, respectively. Our cohort consisted of 2129 TAVR procedures performed at 10 hospitals. The observed mortality was 7.0% at 30 days and 16.4% at 1 year, with a range of 4% to 10% and 8% to 22%, respectively, across hospitals. We developed case-mix adjustment models using 28 clinically relevant variables. Using 30-day and 1-year RSMR to profile each hospital, we found that all hospitals performed as expected, with 95% CI that included the provincial average. We found no significant interhospital variation in RSMR among hospitals, suggesting that quality improvement efforts should be directed at aspects other than the variation in observed mortality.

PROFILING HOSPITAL PERFORMANCE BASED ON MORTALITY AFTER TRANS-CATHETER AORTIC VALVE REPLACEMENT IN ONTARIO, CANADA

Public reporting of hospital-level outcomes is increasingly common as a means to target quality improvement strategies to ensure the delivery of optimal care. Despite the rapid dissemination of transcatheter aortic valve replacement (TAVR), there is a paucity of reliable case-mix adjustment models for hospital profiling in TAVR. Our objective was to develop models for calculating risk-standardized all-cause mortality rates (RSMR) post-TAVR. In this population-based study in Ontario, Canada, we identified all patients that underwent a TAVR procedure between April 1, 2012 and March 31, 2016. For each hospital, we calculated 30-day and 1-year RSMR, using 2-level hierarchical logistic regression models that accounted for patient-specific demographic and clinical characteristics as well as the clustering of patients within the same hospital using a hospital-specific random effects. We classified each hospital into one of three groups: performing worse than expected, better than expected, or performing as expected, based on whether the 95% confidence interval (CI) of the RSMR was above, below or included the provincial average mortality rate respectively. Our cohort consisted of 2,129 TAVR procedures performed at 10 hospitals. The observed mortality was 7.0% at 30-days and 16.4% at one-year with a range of 4%-10% and 8%-22% respectively across hospitals. We developed case-mix adjustment models using 28 clinically relevant, available variables. Using 30-days and one-year RSMR to profile each hospital, we found that all hospitals performed as expected, with 95% CI that included the provincial average. We found no significant inter-hospital variation in RSMR among hospitals, suggesting that quality improvement efforts should be directed at aspects other than the variation in observed mortality.View Large Image Figure ViewerDownload Hi-res image Download (PPT)

Case-Mix Adjustment of the Bereaved Family Survey.

Surveys of bereaved family members are increasingly being used to evaluate end-of-life (EOL) care and to measure organizational performance in EOL care quality. The Bereaved Family Survey (BFS) is used to monitor EOL care quality and benchmark performance in the Veterans Affairs (VA) health-care system. The objective of this study was to develop a case-mix adjustment model for the BFS and to examine changes in facility-level scores following adjustment, in order to provide fair comparisons across facilities. We conducted a cross-sectional secondary analysis of medical record and survey data from veterans and their family members across 146 VA medical centers. Following adjustment using model-based propensity weighting, the mean change in the BFS-Performance Measure score across facilities was -0.6 with a range of -2.6 to 0.6. Fifty-five (38%) facilities changed within ±0.5 percentage points of their unadjusted score. On average, facilities that benefited most from adjustment cared for patients with greater comorbidity burden and were located in urban areas in the Northwest and Midwestern regions of the country. Case-mix adjustment results in minor changes to facility-level BFS scores but allows for fairer comparisons of EOL care quality. Case-mix adjustment of the BFS positions this National Quality Forum-endorsed measure for use in public reporting and internal quality dashboards for VA leadership and may inform the development and refinement of case-mix adjustment models for other surveys of bereaved family members.

Patient-reported outcome measures in arthroplasty registries

— The International Society of Arthroplasty Registries (ISAR) Patient-Reported Outcome Measures (PROMs) Working Group have evaluated and recommended best practices in the selection, administration, and interpretation of PROMs for hip and knee arthroplasty registries. The 2 generic PROMs in common use are the Short Form health surveys (SF-36 or SF-12) and EuroQol 5-dimension (EQ-5D). The Working Group recommends that registries should choose specific PROMs that have been appropriately developed with good measurement properties for arthroplasty patients. The Working Group recommend the use of a 1-item pain question (“During the past 4 weeks, how would you describe the pain you usually have in your [right/left] [hip/knee]?”; response: none, very mild, mild, moderate, or severe) and a single-item satisfaction outcome (“How satisfied are you with your [right/left] [hip/knee] replacement?”; response: very unsatisfied, dissatisfied, neutral, satisfied, or very satisfied). Survey logistics include patient instructions, paper- and electronic-based data collection, reminders for follow-up, centralized as opposed to hospital-based follow-up, sample size, patient- or joint-specific evaluation, collection intervals, frequency of response, missing values, and factors in establishing a PROMs registry program. The Working Group recommends including age, sex, diagnosis at joint, general health status preoperatively, and joint pain and function score in case-mix adjustment models. Interpretation and statistical analysis should consider the absolute level of pain, function, and general health status as well as improvement, missing data, approaches to analysis and case-mix adjustment, minimal clinically important difference, and minimal detectable change. The Working Group recommends data collection immediately before and 1 year after surgery, a threshold of 60% for acceptable frequency of response, documentation of non-responders, and documentation of incomplete or missing data.

Accountability of Hospitals for Medicare Beneficiaries' Postacute Care Discharge Disposition.

Accountability of Hospitals for Medicare Beneficiaries' Postacute Care Discharge Disposition.

Capsule Commentary on Shih et al., Functional Status Outperforms Comorbidities in Predicting Acute Care Readmissions in Medically Complex Patients.

We know very intimate details about our patients—the pressure inside their right ventricle, the concentration of ions in their blood, and the diameter of their blood vessels. But we often don’t know if they can walk, think, or feed themselves. For decades, we have known that functional status predicts outcomes of persons in their homes, in hospitals, and health care facilities. Moreover, many people see functional autonomy as an endpoint in its own right, not just as a predictor of adverse outcomes. Measuring functional status in hospitalized individuals is not particularly difficult or novel, yet it is a very strong predictor of in-hospital death. Recent reports also indicate that it predicts readmission after discharge from acute care. Shih et al. go on to show that functional status predicts 30-day readmission in patients recently discharged from rehabilitation facilities. While not completely accurate, models incorporating functional status measures were found to be superior to models incorporating only demographic and disease-based measures. Considering other factors—notably social networks and supports, social resources, and adequacy of follow-up care—may further improve accuracy. While this is hardly surprising, it is important. First, casemix adjustment models for readmission will be highly inaccurate if they do not consider functional capacity, and this inaccuracy could distort incentives to providers. Second, inpatient care models which reduce the risk of functional decline may also reduce the risk of readmission. Third, we should consider targeting individuals with greater disability for earlier and more intensive post-discharge follow-up. Further study into functional trajectories is also needed. Functional status is a dynamic process, dependent on a variety of factors. Patients with improving functional status may be at less risk than those with deteriorating function. The role of functional status in determining other longer-term outcomes, such as death after discharge, should also be examined. However, we should not wait on the results of these studies. We must start measuring functional status, and we should adopt hospital care models which prevent functional decline sooner rather than later.

PTH-144 Risk adjustment for surgical short-term outcomes for oesophago-gastric cancer patients

Introduction Health outcomes are increasingly compared between providers with the aim to benchmark quality of care. In such benchmarks adjusting for patient characteristics is of major importance to avoid unfair comparisons between providers. We developed a case-mix adjustment model for comparing 30-, 90-day mortality and anastomotic leakage rates after oesophagus gastric (O-G) cancer resections between NHS hospitals. Method We reviewed the literature for existing prediction models for 30-, 90-day mortality and anastomotic leakage after oesophago-gastric cancer resections. Based on this review we selected predictors that were consequently used to develop a case-mix adjustment model in the national audit data of 4,882 O-G cancer patients undergoing surgery in the years 2012 and 2013. Models were developed with logistic regression analysis. Discriminative ability was quantified with the Area under the ROC curve and internal validity was assessed using a bootstrap procedure. Results The risk prediction models identified in the literature were not suitable for case-mix adjustment in the NHS audit data as they required detailed laboratory data that are not available at the national level. Moreover, the majority of existing models were methodologically weak and developed based on data pooled over long periods of time during which outcomes improved considerably in the UK. In the data used for model development, 30-day mortality was 2.3%, 90-day mortality was 4.4% and 6.2% of the patients developed an anastomotic leakage. Discriminative ability in the model development dataset was 0.70, 0.69 and 0.63 for 30-day mortality, 90-day mortality and anastomotic leakage, respectively. The internally validated AUCs were well calibrated and showed moderate discriminative ability (area under ROC curve 0.65 for 30-day mortality, 0.66 for 90-day mortality, yet only 0.59 for anastomotic leakage). Conclusion Based on nationally available data, we developed three case mix adjustment models for postoperative outcomes in the largest contemporary cohort of O-G patients undergoing curative surgery. These models should be used for the risk adjustment when assessing hospital performance in the NHS. Disclosure of interest None Declared.

WS18.2 Using funnel plots to make meaningful centre comparisons

Objectives The UK CF Registry annual reports include comparisons between centres on key outcomes such as FEV 1 using rankings. While illustrating the distribution between centres, they promote the assumption that those with the highest measures provide better care. We hypothesised a more scientific approach based on statistical process control and adjustment for case-mix may identify exceptional CF care services in terms of clinically meaningful outcomes. Methods We extracted FEV 1 and BMI on patients aged 12–21 from annual review data (2007–2012). Funnel plots were generated with confidence limits at 2 and 3 standard deviations (SD). Centres with mean values outside these limits are said to display cause – variability outside what one would expect. Outcomes were then adjusted for case mix (including gender, genotype, pancreatic sufficiency and socio-economic deprivation) and analysed using funnel plots. Results Using a primary outcome of FEV 1 at age 15, data for 1066 patients from 31 paediatric centres were available. Funnel plots of unadjusted data showed 8 centres with evidence of special cause variation (at 2SD limits). Initial adjustment for case-mix reduced the number of centres outside these limits to 3. None of the centres were outside the 3SD limits in either analysis. Similar patterns were found for BMI percentile in this age group. Conclusion In conclusion the work to-date illustrates that funnel plots can be used to identify differences in FEV 1 and BMI between specialist paediatric centres. Case-mix adjustment models should develop into a useful tool for making centre comparisons which can continue to be used by stakeholders.

Comparing colon cancer outcomes: The impact of low hospital case volume and case-mix adjustment

ObjectiveWhen comparing performance across hospitals it is essential to consider the noise caused by low hospital case volume and to perform adequate case-mix adjustment. We aimed to quantify the role of noise and case-mix adjustment on standardized postoperative mortality and anastomotic leakage (AL) rates. MethodsWe studied 13 120 patients who underwent colon cancer resection in 85 Dutch hospitals. We addressed differences between hospitals in postoperative mortality and AL, using fixed (ignoring noise) and random effects (incorporating noise) logistic regression models with general and additional, disease specific, case-mix adjustment. ResultsAdding disease specific variables improved the performance of the case-mix adjustment models for postoperative mortality (c-statistic increased from 0.77 to 0.81). The overall variation in standardized mortality ratios was similar, but some individual hospitals changed considerably. For the standardized AL rates the performance of the adjustment models was poor (c-statistic 0.59 and 0.60) and overall variation was small. Most of the observed variation between hospitals was actually noise. ConclusionNoise had a larger effect on hospital performance than extended case-mix adjustment, although some individual hospital outcome rates were affected by more detailed case-mix adjustment. To compare outcomes between hospitals it is crucial to consider noise due to low hospital case volume with a random effects model.

Can valid and practical risk-prediction or casemix adjustment models, including adjustment for comorbidity, be generated from English hospital administrative data (Hospital Episode Statistics)? A national observational study

BackgroundNHS hospitals collect a wealth of administrative data covering accident and emergency (A&amp;E) department attendances, inpatient and day case activity, and outpatient appointments. Such data are increasingly being used to compare units and services, but adjusting for risk is difficult.ObjectivesTo derive robust risk-adjustment models for various patient groups, including those admitted for heart failure (HF), acute myocardial infarction, colorectal and orthopaedic surgery, and outcomes adjusting for available patient factors such as comorbidity, using England’s Hospital Episode Statistics (HES) data. To assess if more sophisticated statistical methods based on machine learning such as artificial neural networks (ANNs) outperform traditional logistic regression (LR) for risk prediction. To update and assess for the NHS the Charlson index for comorbidity. To assess the usefulness of outpatient data for these models.Main outcome measuresMortality, readmission, return to theatre, outpatient non-attendance. For HF patients we considered various readmission measures such as diagnosis-specific and total within a year.MethodsWe systematically reviewed studies comparing two or more comorbidity indices. Logistic regression, ANNs, support vector machines and random forests were compared for mortality and readmission. Models were assessed using discrimination and calibration statistics. Competing risks proportional hazards regression and various count models were used for future admissions and bed-days.ResultsOur systematic review and empirical analysis suggested that for general purposes comorbidity is currently best described by the set of 30 Elixhauser comorbidities plus dementia. Model discrimination was often high for mortality and poor, or at best moderate, for other outcomes, for examplec = 0.62 for readmission andc = 0.73 for death following stroke. Calibration was often good for procedure groups but poorer for diagnosis groups, with overprediction of low risk a common cause. The machine learning methods we investigated offered little beyond LR for their greater complexity and implementation difficulties. For HF, some patient-level predictors differed by primary diagnosis of readmission but not by length of follow-up. Prior non-attendance at outpatient appointments was a useful, strong predictor of readmission. Hospital-level readmission rates for HF did not correlate with readmission rates for non-HF; hospital performance on national audit process measures largely correlated only with HF readmission rates.ConclusionsMany practical risk-prediction or casemix adjustment models can be generated from HES data using LR, though an extra step is often required for accurate calibration. Including outpatient data in readmission models is useful. The three machine learning methods we assessed added little with these data. Readmission rates for HF patients should be divided by diagnosis on readmission when used for quality improvement.Future workAs HES data continue to develop and improve in scope and accuracy, they can be used more, for instance A&amp;E records. The return to theatre metric appears promising and could be extended to other index procedures and specialties. While our data did not warrant the testing of a larger number of machine learning methods, databases augmented with physiological and pathology information, for example, might benefit from methods such as boosted trees. Finally, one could apply the HF readmissions analysis to other chronic conditions.FundingThe National Institute for Health Research Health Services and Delivery Research programme.

Ranking and Rankability of Hospital Postoperative Mortality Rates in Colorectal Cancer Surgery

To examine to what extent random variation and variation in case-mix influence hospital rankings on the basis of mortality rates and to determine the suitability of mortality for ranking hospitals in colorectal surgery. Comparing and ranking postoperative mortality rates between hospitals becomes increasingly popular. Differences in hospital case-mix, and chance variation related to caseload, may influence rankings. The suitability of mortality for rankings remains unclear. Data were derived from the Dutch Surgical Colorectal Audit. Hospital rankings based on fixed- and random-effects logistic regression models, unadjusted and adjusted for case-mix were compared with the percentile based on expected ranks (the chance that a hospital performs better than a random hospital). Rankability, measuring which part of variation between hospitals is not due to chance, was calculated. Some 25,591 patients undergoing colorectal resections in 92 hospitals were evaluated. Postoperative mortality rates ranged between 0% and 8.8%. Adjustment for case-mix with a fixed-effects model caused large changes in rankings. A smaller additional effect on changes in rankings occurred after adjusting with a random-effects model, with lower volume hospitals moving toward the mean. Percentile based on expected ranks ranged between 10% and 85%. Rankability was 38%, meaning that 62% of hospital variation in mortality was due to chance. Hospital ranks changed after case-mix adjustment and random-effects models, compared with unadjusted analysis. A large proportion of hospital variation in mortality was due to chance. Caution should be warranted when interpreting hospital rankings on the basis of postoperative mortality. Percentiles of expected ranks may help identify hospitals with exceptional performance.

The 30-day versus in-hospital and 90-day mortality after esophagectomy as indicators for quality of care.

To describe causes of death in the first year after esophagectomy and determine the time frame that should be used for measurement of quality of surgery. A case-mix adjustment model was developed for the comparison between hospitals. The time period in which postoperative mortality should be measured as a performance indicator is debated. Cause of death was identified for patients in a tertiary referral hospital who died within 1 year after surgery and classified as surgery related or not surgery related. Sensitivity and specificity for detecting deaths related to surgery were calculated for different periods of follow-up. Case-mix adjustment models for 30-day mortality (30DM), in-hospital mortality, and 90-day mortality (90DM) were developed. In total, 1282 patients underwent esophagectomy. 30DM was 2.9%, the in-hospital mortality rate was 5.1% and 90DM was 7%. Beyond 30 days, a substantial number of deaths were related to the operation, especially due to anastomotic leakage. Postdischarge nononcological mortality was most frequently caused by sudden death. One in 5 patients died because of recurrent disease, being the most important threat in the first year after surgery. The 30DM had a sensitivity for detecting surgery-related deaths of 33% and a specificity of 100%. The 90DM had a sensitivity of 74% and a specificity of 96%. A period of postoperative follow-up longer than 30 days needs to be considered when comparing surgical performance between institutes. In the case-mix adjustment model for 90DM, no other variables have to be taken into account compared to those involved in 30DM.

Direct risk standardisation: a new method for comparing casemix adjusted event rates using complex models

BackgroundComparison of outcomes between populations or centres may be confounded by any casemix differences and standardisation is carried out to avoid this. However, when the casemix adjustment models are large and complex, direct standardisation has been described as “practically impossible”, and indirect standardisation may lead to unfair comparisons. We propose a new method of directly standardising for risk rather than standardising for casemix which overcomes these problems.MethodsUsing a casemix model which is the same model as would be used in indirect standardisation, the risk in individuals is estimated. Risk categories are defined, and event rates in each category for each centre to be compared are calculated. A weighted sum of the risk category specific event rates is then calculated. We have illustrated this method using data on 6 million admissions to 146 hospitals in England in 2007/8 and an existing model with over 5000 casemix combinations, and a second dataset of 18,668 adult emergency admissions to 9 centres in the UK and overseas and a published model with over 20,000 casemix combinations and a continuous covariate.ResultsSubstantial differences between conventional directly casemix standardised rates and rates from direct risk standardisation (DRS) were found. Results based on DRS were very similar to Standardised Mortality Ratios (SMRs) obtained from indirect standardisation, with similar standard errors.ConclusionsDirect risk standardisation using our proposed method is as straightforward as using conventional direct or indirect standardisation, always enables fair comparisons of performance to be made, can use continuous casemix covariates, and was found in our examples to have similar standard errors to the SMR. It should be preferred when there is a risk that conventional direct or indirect standardisation will lead to unfair comparisons.

A systematic review of case-mix adjustment models for stroke

Objective: To identify any externally validated prognostic model for predicting outcome in unselected populations following acute stroke comprising variables feasible for collection in routine care. Data sources: Searches were run in MEDLINE, EMBASE, CINAHL, PsycInfo, AMED and ISI Web of Science with no limits on publication date or language. Review methods: Any study describing the development or external validation of a discernible prognostic model to predict any valid outcome following acute stroke was included. Papers were retained if they met pre-specified inclusion criteria identified from previous reviews and pertinent discussion papers. Data extraction focused on methodological quality of model development, generalizability and feasibility of variable collection. Model performance was examined through consideration of external validation studies. Results: Seventeen externally validated models were identified from 43 papers fulfilling inclusion criteria. Quality of studies describing model development was variable and model performance in external validation studies was generally poor. Models were generally constructed through secondary use of randomized trial or stroke database data. Prognostic variables broadly encompassed markers of stroke severity, pre-stroke function and comorbidities. One model that fulfilled the review criteria and had extensive external validation in a range of post-stroke populations was identified (the Six Simple Variables model). Conclusion: The Six Simple Variables model performed well in six external validation studies, although prediction of outcome in patients with milder strokes was less reliable. Other models identified in this review have been developed using robust methodology but comprise more complex clinical variables which may limit their utility in routine stroke care.