A mortality risk model to adjust for case mix in UK paediatric cardiac surgery

Abstract

BackgroundCongenital heart disease (CHD) is a relatively common disorder in childhood, affecting approximately 8–9 per 1000 live-born infants annually in the UK. CHD often involves serious abnormalities and is an important cause of childhood mortality, morbidity and disability. It is generally recognised that it is important and valuable to monitor outcomes in cardiac surgery and that, to do so fairly and effectively, one needs to risk stratify the case load of each unit. There is evidence that, since outcome monitoring in adult cardiac surgery became mandatory and routine, outcomes have improved. At present, no process for routinely monitoring risk-adjusted outcomes in paediatric cardiac surgery exists.ObjectivesTo establish whether or not a risk model can be developed that is fit for the purpose of adjusting for case mix severity to facilitate routine monitoring of outcomes for paediatric cardiac surgery in the UK and to assess whether or not and how diagnostic information can augment procedural information in risk adjustment.MethodsData from the Central Cardiac Audit Database (CCAD) for all cardiac surgery procedures, excluding reoperations within 30 days, performed in the UK for patients < 16 years between 2000 and 2010 (38,597 patient episodes) were included: 70% for model development and 30% quarantined for validation. The outcome was 30-day survival, as supplied to CCAD through the Central Register of NHS patients (now the Medical Research Information Service). The CCAD defines 36 ‘specific procedures’. Nine of these were merged as a ‘low-volume specific procedure’ group (< 90 cases each in the entire development set). Unassigned cases were grouped as ‘not a specific procedure’. Twenty-four ‘primary’ cardiac diagnoses and separately a categorisation of ‘univentricular’ status were defined using a hierarchical algorithm developed by the study team based on International Paediatric and Congenital Cardiac codes. Comorbidities considered included prematurity (< 37 weeks' gestation), Down syndrome, constellations of features that constitute a recognised syndrome, congenital structural defects of organs other than the heart and acquired conditions. Other candidate variables included use of bypass and patient age, weight and sex. Data were analysed using logistic regression.ResultsIn the development set, there were 25,665 episodes that resulted in survival to 30 days, 693 episodes for which the vital status at 30 days was unknown and 854 episodes that resulted in death within 30 days in the development set (mortality 3.2% overall). The risk model developed includes the following factors: specific procedure, primary cardiac diagnosis grouped into low-, medium- and high-risk categories, univentricular heart status, age band (neonate, infant, child), continuous age, continuous weight, presence of a comorbidity other than Down syndrome and use of bypass. To account for decreasing mortality over time in the development set, a binary indicator for operations performed after 1 January 2007 is also included in the model. We were able to calculate a risk score for 95% of cases in the test set: weight was missing in 5% of cases. Data completeness improved over time. The proposed model discriminated well: the area under the receiver operating characteristic curve (AUC) for the test set was 0.77 (0.81 for post-2007 data). Removal of all but procedural information gave a reduced AUC of 0.72. The model performed well across the spectrum of predicted risk in the entire data set, but there was underestimation of mortality risk in the test set among neonates operated from 2007.LimitationsAn important limitation is that the model pertains to short-term 30-day outcomes (not long-term outcomes) and is designed for the purpose of routine monitoring for quality assurance rather than bedside-type predictions for individual patients. Over the recent period in the validation set (since 2007), the model was found to underestimate risk at the very high-risk end (> 10% risk), in particular among neonates. This indicates that risk adjustment based on the current parameterisation of the model will potentially give an unduly negative impression of outcomes at those centres with a high proportion of high-risk cases. Finally, any risk model used for ongoing quality improvement initiatives needs to be regularly updated as data quality improves and clinical practice evolves.ConclusionsFor the first time diagnostic information has been successfully incorporated into risk adjustment for short-term outcomes in this patient group, which added discriminatory power. The risk model is fit for purpose, although the underestimation of risk in recent neonates is an important caveat. Several centres have expressed an interest in piloting the risk model and the accompanying monitoring tool. Future work includes developing software to generate variable life-adjusted display charts within units using the risk model; using the risk model to explore trends in case mix over time; and informing future work in evaluating long-term outcomes for children with CHD.FundingThe National Institute for Health Research Health Services and Delivery Research programme.