Abstract
Two assumptions of speaking proficiency tests are that the speech produced is spontaneous and the the scores on those tests predict what examinees can do in real-world communicative situations. Therefore, when examinees memorize scripts for their oral responses, the validity of the score interpretation is threatened. While the American Council on the Teaching of Foreign Languages (ACTFL) Proficiency Guidelines identify rehearsed content as a major hindrance to interviewees being rated above Novice High, many examinees still prepare for speaking tests by memorizing and rehearsing scripts hoping these "performances" are awarded higher scores. To investigate this phenomenon, researchers screened 300 previously rated Oral Proficiency Interview-computer (OPIc) tests and found 39 examinees who had at least one response that had been tagged as rehearsed. Each examinee’s responses were then transcribed, and the spontaneous and rehearsed tasks were compared. Temporal fluency articulation rates differed significantly between the spontaneous and rehearsed segments; however, the strongest evidence of memorization lay in the transcriptions and the patterns that emerged within and across interviews. Test developers, therefore, need to be vigilant in creating scoring guidelines for rehearsed content.
Highlights
Globalization has led to an increased demand for English skills
Research question 1: differences in temporal fluency features To determine whether there was a significant difference between the mean length of utterances (MLU), silent pauses (SP), and articulation rate (AR) of spontaneous versus rehearsed segments, a paired samples t test was utilized to compare the sample means
There was no significant difference between the spontaneous and rehearsed segments in terms of MLU (Table 3, Fig. 2) or SP (Table 3, Fig. 3)
Summary
Globalization has led to an increased demand for English skills. Some companies, even those founded and based in countries where English is not an official or common language, are enforcing language policies to make themselves more competitive and better communicators in a global market through requiring English tests. One assumption of speaking proficiency tests is that the speech produced is spontaneous so that the interpretation of the scores on those tests will be predictive of what examinees can do in these real-world communicative situations. When examinees memorize scripts for their oral responses, validity is threatened. For tests to be valid, responses need to be consistently rated, Gates et al Language Testing in Asia (2020) 10:18 and score interpretations need to be both meaningful and impartial (Kelly et al 2017); when raters encounter memorized responses intermingled with the spontaneous, scoring can be impacted. The prevalence of rehearsed responses in speaking tests should be evaluated to determine the effect it might have on test validity
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
