Validation of a Chinese online placement test

Abstract

Situated at the intersection of theories of second language (L2) reading, cognition, and assessment, the study in this chapter explores how to develop and validate a Chinese placement test through action research. The chapter first presents the process of developing the placement test. Three levels of proficiency needed to be separated through the placement test: Beginning, Intermediate, and Advanced. The language skills and knowledge assessed in the placement test included vocabulary, grammar, and reading comprehension. Listening and speaking abilities were excluded in this placement test because some test takers may speak Chinese at home, but have never studied Chinese literacy. The format of the test was multiple-choice questions. Altogether, there were 125 items of questions, covering proficiency levels from Novice-low to Intermediate-high by the ACTFL standards. Then, this chapter discusses the procedures of administering the test and validating its authenticity. Various data from 56 students were included in the analysis. The validation examined the validity and reliability of the test, and was carried out both quantitatively and qualitatively. Two analyses of validity were conducted. The first examined test content in relation to curriculum content. Three external raters were asked to rate the content of the test. The results indicate that their rating was highly consistent. The second analysis compared the placement test scores with the students’ midterm exams. The correlation coefficients reveal that there was a moderate relationship between the placement test and students’ midterm examination scores. The coefficients also demonstrate that the placement test had a strong relationship with the grammar and text comprehension sections in the students’ midterm examinations. Reliability coefficients were examined. The results showed the reliability of the placement test was high, indicating the test is fairly reliable. Qualitative data was obtained through a post-test survey questionnaire and semi-structured interviews. Through the questionnaire, it was found that 53 out of 56 students considered that it was a fair test of their literacy ability. Follow-up semi-structured interviews were conducted with seven test takers covering their concerns about the test in terms of layout, structure, difficulty level, item type, and content. These interviews provided supplementary data on test takers’ perceptions and helped highlight areas for further improvement. Overall, the results show that the placement test was valid and could successfully assess the learners’ literacy abilities at the different proficiency levels. In the last part of the paper, some suggestions have been provided to improve the test’s validity and reliability. This study strengthens validity arguments and enhances understanding of the complexity of Chinese placement test construction.