A Bayesian hierarchical approach to comparative audit for carotid surgery

Abstract

Objectives: the aim of this study was to illustrate how a Bayesian hierarchical modelling approach can aid the reliable comparison of outcome rates between surgeons. Design: retrospective analysis of prospective and retrospective data. Materials: binary outcome data (death/stroke within 30 days), together with information on 15 possible risk factors specific for CEA were available on 836 CEAs performed by four vascular surgeons from 1992–99. The median patient age was 68 (range 38–86) years and 60% were men. Methods: the model was developed using the WinBUGS software. After adjusting for patient-level risk factors, a cross-validatory approach was adopted to identify “divergent” performance. A ranking exercise was also carried out. Results: the overall observed 30-day stroke/death rate was 3.9% (33/836). The model found diabetes, stroke and heart disease to be significant risk factors. There was no significant difference between the predicted and observed outcome rates for any surgeon (Bayesian p-value > 0.05). Each surgeon had a median rank of 3 with associated 95% CI 1.0–5.0, despite the variability of observed stroke/death rate from 2.9–4.4%. After risk adjustment, there was very little residual between-surgeon variability in outcome rate. Conclusions: Bayesian hierarchical models can help to accurately quantify the uncertainty associated with surgeons' performance and rank.