Comparability of a paper-based language test and a computer-based language test

Abstract

With the advent of the digital revolution, language testers have endeavored to utilize state-of-the-art computer technology to satisfy the ever-growing need for a tool to measure English communication skills with maximal accuracy and efficiency. Thanks to the concerted efforts made by experts in such fields as computational linguistics, computer engineering, computer-assisted language learning, and psychometrics, language testers have recently succeeded in developing computer/web-based language tests. Among them are the TOEFL CBT by Educational Testing Service and CommuniCAT by the University of Cambridge Local Examinations Syndicate. As with the paper-based language test (PBLT), more rigorous research is now being conducted on the validity of computer-based language tests (CBLT) and computer adaptive language tests (CALT). Content analyses and comparability studies of PBLT and CBLT/CALT are prerequisites to such validation research. In this context, utilizing an EFL test battery entitled the Test of English Proficiency developed by Seoul National University (TEPS), the present study is aimed at addressing the issue of the comparability between PBLT and CBLT based on content and construct validation employing content analyses based on corpus linguistic techniques in addition to such statistical analyses as correlational analyses, ANOVA, and confirmatory factor analyses. The findings support comparability between the CBLT version and the PBLT version of the TEPS subtests (listening comprehension, grammar, vocabulary, and reading comprehension) in question.