숙련 및 미숙련 평가자들의 총체적, 분석적 및 진단적 채점기준을 활용한 영작문 평가 방식의 비교 분석 연구

Abstract

Objectives This study aimed to investigate experienced and novice raters’ assessment behaviors when they used holistic, analytic, and diagnostic rating scales in ESL writing assessments. It also closed examined how the two groups of raters perceived the different rating scales.
 Methods The experienced and novice rater groups consisted of five native English-speaking professors with more than three years of experience teaching and assessing ESL writing and four native English-speaking professors with less than three years of experience teaching and assessing ESL writing. Both groups of raters assessed 30 ESL compositions using the three different rating scales and completed an assessor questionnaire.
 Results The findings indicated that the novice raters were more lenient in assessing ESL writing across the three rating scales. The experienced raters also showed more variance in their scores than the novice raters when they used the holistic and analytic (i.e., task fulfillment) rating scales. In contrast, the novice raters showed more variance in their scores than the experienced raters when they used the analytic (i.e., lexical and grammar) and diagnostic rating scales. While the experienced raters preferred the holistic rating scale, the novice raters preferred the analytic rating scale. Both groups liked the diagnostic rating scale the least.
 Conclusions Based on these findings, this study proposes that raters should consider the assessment purpose and context and rater variability when choosing an appropriate rating scale.