Abstract
<p>This study attempts to validate an academic group tutorial discussion speaking test for undergraduate freshmen students taking initial EAP training at a university in Hong Kong in terms of task, rater and criterion validity. Three quantitative measures (Cronbach’s Alpha, Intraclass Correlation Coefficient, and Exploratory Factor Analysis) are used to assess validity of rater scores for the test using a rubric with considerations for assessment of academic stance presentation, inter-candidate interaction, and individual language proficiency. These results are triangulated with post-hoc interview data from the raters regarding the difficulties they face assessing individual proficiency and group interaction over time. The results suggest that current provisions of the rubric in dealing with the assessment of interaction in group settings (namely visual cues such as “active listening” as well as provisions for interruptions in the form of “domination”) are problematic, and that raters are unable to separate the grading of academic stance from the grading of language concerns. We also note affective and cognitive difficulties involved with assessing extended periods of interactional discourse including student talking time (or lack of it), the group dynamic, and raters” personal beliefs and practice as threats to validity that the statistical measurements were unable to capture. A new sample rubric and further suggestions for improving the validity of group tutorial assessments are provided.</p>
Highlights
1.1 Use of Group Oral Assessment in Second Language ContextsThere is a growing trend in the use of peer-to-peer or group oral language proficiency assessments across international and course-specific assessments devised by local teachers (Ducasse & Brown, 2009)
The results suggest that current provisions of the rubric in dealing with the assessment of interaction in group settings are problematic, and that raters are unable to separate the grading of academic stance from the grading of language concerns
Criteria Stance Explain academic concepts Argue for stance with sources Critically respond to others” stance Interaction Never dominate the discussion Never read from your notes Link contributions to what has been said before Use active listening skills Language Always comprehensible Mistakes of grammar/vocab Fluency
Summary
There is a growing trend in the use of peer-to-peer or group oral language proficiency assessments across international and course-specific assessments devised by local teachers (Ducasse & Brown, 2009). Such practice is encouraged from a number of perspectives, including time/cost savings on testing multiple students in one session, the ability of students to engage with Englishes other than the standard variety (Kirkpatrick, 2007), and the avoidance of memorized “interview” talk based on predictable questions (Van Lier, 1989) with unbalanced power relationships between interviewee-interviewer leading to a lack of opportunity for participants to go beyond question-answer conversational structures (Lazeraton, 1992; Johnson, 2001; Taylor, 2001; Galaczi, 2004).
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
