The hunt for “indigenous criteria” in assessing communication in the physiotherapy workplace

Abstract

Gaining insights from domain experts into how they view communication in real world settings is recognized as an important authenticity consideration in the development of criteria to assess language proficiency for specific academic or occupational purposes. These “indigenous” criteria represent an articulation of the test construct and should therefore reflect what is germane to the particular domain of language use rather than general language-focused criteria familiar from other language tests. The methodological question of how to elicit such insights is, however, complex and has been addressed by various researchers using different methodological and theoretical frameworks. The paper draws on data from a larger research project to explore the affordances and constraints of more or less direct approaches to eliciting domain experts’ perspectives on what matters for effective communication in the workplace. The domain experts in this case were physiotherapy educators and supervisors. The study offers a qualitative comparison of expert feedback gathered from three different sites. Two were in the workplace where the communication skills of physiotherapy students in training were assessed routinely and the feedback given to them was naturally occurring rather than elicited. The third was a more artificial workshop setting in which video-recorded interactions between student and patients or simulated patients (i.e., actors role-playing a patient) were shown to two groups of expert informants who were then asked by the researcher to comment on the strengths and weaknesses of each performance. A qualitative analysis revealed that the nature of expert feedback differed significantly at each site, with the routinely occurring feedback containing scant and vague reference to language and communication aspects. The workshop setting, although it was less authentic, yielded much richer insights into the physiotherapists’ views about workplace communication. The implications of our findings for the development of relevant language test criteria are considered.