Abstract
Assessment of foreign/second language (L2) oral proficiency is known to be complex and influenced by the local context. In Sweden, extensive assessment guidelines for the National English Speaking Test (NEST) are offered to teachers, who act as raters of their own students’ performances on this high-stakes L2 English oral proficiency (OP) test. Despite guidelines, teachers commonly construct their own NEST scoring rubric. The present study aims to unveil teachers-as-raters’ conceptualizations, as these emerge from the self-made scoring rubrics, and possible transformations of policy. Data consist of 20 teacher-generated scoring rubrics used for assessing NEST (years 6 and 9). Rubrics were collected via personal networks and online teacher membership groups. Employing content analysis, data were analysed qualitatively to examine (i) what OP sub-skills were in focus for assessment, (ii) how sub-skills were conceptualized, and (iii) scoring rubric design. Results showed that the content and design of rubrics were heavily influenced by the official assessment guidelines, which led to broad consensus about what to assess—but not about how to assess. Lack of consensus was particularly salient for interactive skills. Analysis of policy transformations revealed that teachers’ self-made templates, in fact, lead to an analytic rather than a holistic assessment practice.
Highlights
Oral proficiency in a second and/or foreign language (L2) is “at the very heart of what it means to be able to use a foreign language” (Alderson and Bachman 2004, p. ix), but it is the language skill that is the most difficult to assess in a reliable way.One of the challenges raters of L2 oral proficiency face is the fact that numerous aspects of quality need to be considered simultaneously by raters (Bachman 1990)
The present study aims to contribute to filling this gap of knowledge, when it comes to what content teachers decide to include in an L2 oral proficiency (OP) assessment rubric
RQ1: What Sub-Skills of Oral Proficiency Are in Focus for Assessment in Teachers’ Own
Summary
Oral proficiency in a second and/or foreign language (L2) is “at the very heart of what it means to be able to use a foreign language” (Alderson and Bachman 2004, p. ix), but it is the language skill that is the most difficult to assess in a reliable way (ibid.).One of the challenges raters of L2 oral proficiency face is the fact that numerous aspects of quality need to be considered simultaneously by raters (Bachman 1990). L2 oral tests (see e.g., Young and He 1998; May 2009), entails a realization that each interaction is unique in terms of context and co-participants and that, as such, generalizations regarding individual proficiency are virtually impossible. Adding to these difficulties, raters will inevitably make different professional judgments of the same candidate’s performance in an interaction-based test, to the extent that some scholars have argued that “aiming for perfect agreement among raters in paired speaking assessment is not feasible” A strive for equity and fair grading in high-stakes contexts is, and should be, of central concern to stakeholders in language testing and assessment
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
