Calibration Bias and the Interpretation of Clinical Learning Environment Perceptions Surveys.

Abstract

The clinical learning environment (CLE) is frequently assessed using perceptions surveys, such as the AAMC Graduation Questionnaire and ACGME Resident/Fellow Survey. However, these survey responses often capture subjective factors not directly related to the trainee's CLE experiences. The authors aimed to assess these subjective factors as "calibration bias" and show how it varies by health professions education discipline, and co-varies by program, patient-mix, and trainee factors. We measured calibration bias using 2011-2017 US Department of Veterans Affairs (VA) Learners' Perceptions Survey data to compare medical students and physician residents and fellows (n = 32 830) with nursing (n = 29 758) and allied and associated health (n = 27 092) trainees. Compared to their physician counterparts, nursing trainees (OR 1.31, 95% CI 1.22-1.40) and allied/associated health trainees (1.18, 1.12-1.24) tended to overrate their CLE experiences. Across disciplines, respondents tended to overrate CLEs when reporting 1 higher level (of 5) of psychological safety (3.62, 3.52-3.73), 1 SD more time in the CLE (1.05, 1.04-1.07), female gender (1.13, 1.10-1.16), 1 of 7 lower academic level (0.95, 1.04-1.07), and having seen the lowest tercile of patients for their respective discipline who lacked social support (1.16, 1.12-1.21) and had low income (1.05, 1.01-1.09), co-occurring addictions (1.06, 1.02-1.10), and mental illness (1.06, 1.02-1.10). Accounting for calibration bias when using perception survey scores is important to better understand physician trainees and the complex clinical learning environments in which they train.