ENGLISH SPEAKING SKILLS ASSESSMENT FOR GRADE 6 THAI STUDENTS: AN APPLICATION OF MULTIVARIATE GENERALIZABILITY THEORY

Abstract

Background and Purpose. This research studies an analytic rating scale for an English speaking skills assessment designed for Grade 6 Thai students learning English as a foreign language. Material and Methods . A rating scale was developed, validated, and then piloted using assessment results from 101 students attending a government school in Bangkok, Thailand. The analytic rating scale developed for the assessment is composed of 6 components: vocabulary, syntax, cohesion, pronunciation, ideational function and fluency. The reliability of the rating scale was examined with different numbers of speaking tasks and raters. Multivariate generalizability theory (G theory) was utilized for the data analysis. Results. The results showed that fluency was the greatest variance component of the composite score of the analytic rating scale, followed by ideational function, cohesion, vocabulary and syntax, and pronunciation respectively. Reliability of the composite score for the speaking analytic rating scale was high (over .80). It was found that the reliability coefficients for each compo- nent would be reliable (over .80) when six or more tasks are used and the number of raters is from 6–10 and above. The dependability increased more when the number of tasks increased and when the number of raters increased. It was also found that a reliable high Phi Coefficient (over .80) could be obtained using only 6 tasks and 3 raters. Conclusion. The main results were then discussed in more detail.