AN INVESTIGATION OF THE IMPACT OF COMPOSITION MEDIUM ON THE QUALITY OF TOEFL WRITING SCORES

Abstract

ABSTRACTThis study examined scores from 133,906 operationally scored Test of English as a Foreign Language™ (TOEFL®) essays to determine whether the choice of composition medium has any impact on score quality for subgroups of test‐takers. Results of analyses demonstrate that (a) scores assigned to word‐processed essays are slightly more reliable than scores assigned to handwritten essays and exhibit higher correlations with TOEFL multiple‐choice subscores; (b) female test‐takers, examinees whose native language is not based on a Roman/Cyrillic alphabet, and examinees with lower English proficiency are more likely to choose the handwriting medium; (c) the probability of choosing handwriting as the composition medium increases with age for Asian examinees, but decreases with age for most European examinees; and (d) examinees with lower TOEFL multiple‐choice scores tend to have higher handwritten than word‐processed essay scores, while examinees with higher TOEFL multiple‐choice scores tend to have similar scores in either medium.