Assessing the assessment: a comparison of the IPASS and ICCAS to measure change in interprofessional competencies following a large-scale interprofessional forum

Abstract

ABSTRACT Measuring the effectiveness of interprofessional education (IPE) experiences is essential but challenging. Surveying learners before (pretest) and after (posttest) an IPE experience may result in minimal change due to response shift bias. Retrospective pretest/posttest assessment may ameliorate response shift bias but may also result in inflated change scores due to social desirability bias. We studied a cohort of 675 students from 12 health professions who completed the Interprofessional Attitudes Scale (IPAS) within the 4 weeks before (pretest) and the 3 weeks after (posttest) an IPE forum and completed the Interprofessional Collaborative Competency Attainment Survey (ICCAS) within the 3 weeks after the forum as a retrospective pretest/posttest. We found higher scores on the pretest IPAS than the retrospective pretest ICCAS and greater change in scores on the ICCAS than the IPAS, indicating potential response shift and social desirability biases. Furthermore, we found few significant correlations between change scores in subscales of the two tests, but a high number of strong and significant correlation among the ICCAS subscale change scores. Our results indicate the timing of pretest administration may impact change scores or that subscales of IPAS may be more unique than those of ICCAS. These findings suggest that educators should consider the potential impact of response shift and social desirability biases when interpreting results of the IPAS and ICCAS in response to an IPE learning experience.