'Lost in translation'?: Developing assessment criteria that value rural practice

Abstract

Rural workforce preparation is often discussed in terms of specific interventions such as rural placements. More technical discussions of education matters seem to belong in the realm of education experts. However, this issues article argues that a focus on quality assessment techniques is important to the rural health agenda. Making connections between the medical education literature and the broader education literature, it explores elements of a qualitative decision-making model as an alternative to narrow competency-based and norm-referenced approaches. In the process it explores assessment techniques that may help educators better translate their intentions to value rural practice into the learning of students. Research suggests that, in Australia at least, many university educators have different and conflicting understanding of assessment criteria. At the same time, the literature on the development of assessment criteria is relatively small in a context in which the medical education literature takes a quantitative, reliability-driven approach. This has important implications for how we ensure that rural practice is given enough emphasis at the level of education that most strongly drives student learning-assessment. This article explores such matters by examining the steps needed to develop assessment criteria in undergraduate medical education courses. It draws on key writings from the past, as well as current debates, in the medical education and broader education literature. It focuses on the detail of assessment techniques to show how the intention to value rural practice can be 'lost in translation' with narrow norm-referenced and competency-based assessment models. Rural health has a stake in technical debates about education in health sciences courses. Like other knowledge and skills, the knowledge and skills important to rural practice cannot be valued at the coalface of student learning if our assessment techniques subvert intentions. Developing the quality of assessment techniques involves scrutiny of not only the medical education literature, but also the broader education literature, including writings about working models of criteria-and-standards-based assessment. This scrutiny suggests assessment techniques are not equal in terms of how well they translate intentions. More than that, it suggests the value to rural health education of shifting from narrow norm-referenced models to best practice in criteria-and-standards-based assessment.