Measuring English vocabulary size via computerized adaptive testing

Abstract

Measuring English vocabulary size in EFL contexts normally requires a large number of test items and relies on paper-and-pencil (P&P) formats. The aim of this study was to examine the feasibility and practicality of computerized adaptive testing (CAT) as an alternative to measuring English vocabulary size. Differing from the fixed, uniform item sequences in conventional P&P tests, CAT adopts a dynamic, adaptive item selection procedure to optimally target the interim ability estimate and reach the convergence, resulting in a shorter, putatively more efficient test-taking process. The study involved three phases. The first phase built up a vocabulary item bank using the Rasch model, which was used for administering the CAT study; the second phase undertook an experiment to compare various termination conditions in both the P&P and CAT contexts; the third phase examined the accuracy and efficiency of the two test modes in classifying test-takers into mastery and non-mastery groups. The results show that testing EFL learners' English vocabulary size with CAT requires only one third of the items in the item bank while still producing comparable vocabulary size estimates to the original test calibrated by all the 180 items in the item bank. The study also demonstrates that CAT can be more efficient and precise in classifying test-takers into mastery and non-mastery groups. These research findings suggest that CAT has great potential in efficiently and precisely measuring EFL learners' English vocabulary size. The relevant research and pedagogical implications are further discussed.