Interpreting multisource feedback: online study of consensus and variation among GP appraisers.

Abstract

GPs collect multisource feedback (MSF) about their professional practice and discuss it at appraisal. Appraisers use such information to identify concerns about a doctor's performance, and to guide the doctor's professional development plan (PDP). To investigate whether GP appraisers detect variation in doctors' MSF results, and the degree of consensus in appraisers' interpretations of this information. Online study of GP appraisers in north-east England. GP appraisers were invited to review eight anonymised doctors' MSF reports, which represented different patterns of scores on the UK General Medical Council's Patient and Colleague Questionnaires. Participants provided a structured assessment of each doctor's report, and recommended actions for their PDP. Appraiser ratings of each report were summarised descriptively. An 'agreement score' was calculated for each appraiser to determine whether their assessments were more lenient than those of other participants. At least one report was assessed by 101/146 appraisers (69%). The pattern of appraisers' ratings suggested that they could detect variation in GPs' MSF results, and recommend reasonable actions for the doctors' PDP. Increasing appraiser age was associated with more favourable interpretations of MSF results. Although preliminary, the finding of broad consensus among GP appraisers in their assessment of MSF reports should be reassuring for GPs, appraisers, and employing organisations. However, if older appraisers are more lenient than younger appraisers in their interpretation of MSF and in the actions they suggest to their appraisees as a result, organisations need to consider what steps could be taken to address such differences.