‘Team observation’: a six-year study of the development and use of multi-source feedback (360-degree assessment) in obstetrics and gynaecology training in the UK

Abstract

Multi-source feedback, or 360-degree assessment, is an important part of the assessment of people in the workplace, in both health and industry. Almost all published work concentrates on content validity and generalizability. However, an assessment system needs construct validity, and has to have practicability and acceptability, without sacrificing fitness for purpose, content validity or inter-rater reliability. This was a six-year study of the first UK-wide hospital-based multi-source feedback system, in the specialty of obstetrics and gynaecology. This paper describes the development of the assessment tool, its use and the analyses of the results in several areas. These are picking up poor performance, congratulating good behaviour, construct validity, the number of domains to be measured, and the minimum number of raters. The study demonstrated that the Team Observation system in reality only measured a very limited number of attributes, and that the main construct under scrutiny is interpersonal behaviour. The system can identify those who may have a problem, using less than 10 raters, and yet the process can be a positive experience for the large majority of people who have been assessed.Practice points•Multi-source feedback (360-degree assessment) is an important tool in assessing the behaviours of doctors in training.•We have developed a simple four domain Team Observation tool that has been used widely within obstetrics and gynaecology training within the UK.•The Team Observation tool may be used for identification of performance regarded as poor and for congratulating performance thought to be excellent in interpersonal behaviours.•Multi-source feedback (360-degree assessment) seems to measure a very limited number of attributes—and the main construct that is measured in the Team Observation system is interpersonal behaviour.