Determining the Number of Patient Charts Necessary for a Reliable Assessment of Practicing Family Physicians’ Performance

Abstract

In many countries, peer assessment programs based on the examination of patient charts are becoming a standard to assess physician's clinical performance. Although data on validity of the process are acceptable, reliability issues need some improvement. This article addresses the rarely studied aspect of optimal number of patient charts for an acceptable reliable assessment. Fifteen patient charts for each of a group of 20 practicing physicians were independently reviewed by 4 professional peer assessors. Generalizability (G) and decision (D) studies were applied to the data. It appears that as few as 10 patient charts are sufficient for any assessor to obtain a G coefficient of 0.80. Results of the current study suggest the possibility of getting generalizable assessments by peer reviewer with minimal information. These results are not in accordance with the concept of case specificity in which it is claimed that performance on a case is a poor predictor of performance on a different case.