Judging the quality of clinical audit by general practitioners: a pilot study comparing the assessments of medical peers and NHS audit specialists

Abstract

Clinical audit informs general practitioner (GP) appraisal and will provide evidence of performance for revalidation in the UK. However, objective evidence is now required. An established peer assessment system may offer an educational solution for making objective judgements on clinical audit quality. National Health Service (NHS) clinical audit specialists could potentially support this system if their audit assessments were comparable with established medical peer assessors. The study aimed to quantify differences between clinical audit specialists and medical peer assessors in their assessments of clinical audit projects. A comparison study of the assessment outcomes of clinical audit reports by two groups using appropriate assessment instruments was conducted. Mean scores were compared and 95% confidence intervals (CIs) and limits of agreement calculated. A two-point mean difference would be relevant. Twelve significant event analysis (SEA) reports and 12 criterion audit projects were assessed by 11 experienced GP assessors and 10 NHS audit specialist novice assessors. For SEA, the mean score difference between groups was <1.0. The 95% CI for bias was -0.1 to 0.5 (P = 0.14). Limits of agreement ranged from -0.7 to 1.2. For criterion audit, a mean score difference of <or=1.0 was calculated for seven projects and scores between 1.1 and 1.9 for four. The 95% CI for bias was 0.8 to 1.5 (P < 0.001). Limits of agreement ranged from -2.5 to -0.0. The study findings suggest that a sample of NHS clinical audit specialists can give numerically accurate feedback scores to GPs on the quality of their clinical audit activity compared with established peer assessors as part of the model outlined.