Are TOEFL iBT® writing test scores related to keyboard type? A survey of keyboard-related practices at testing centers

Abstract

Abstract The strength of a computer-based writing test, such as the TOEFL iBT ® Writing Test, lies in its capability to assess all examinees under the same conditions so that scores reflect the targeted writing abilities rather than differences in testing conditions, such as types of keyboards. The familiarity and proficiency examinees have with a specific type of keyboard could affect their efficiency in writing essays and introduce construct-irrelevant variance, although little research is available in the literature. To explore this, we surveyed 2214 TOEFL iBT testing centers in 134 countries on practices related to keyboard type and analyzed the centers’ responses and the TOEFL iBT scores of examinees from these centers. Results revealed that (a) most testing centers used the U.S. standard English keyboard (USKB) for the test, but a small proportion of centers used a country-specific keyboard (CSKB) after being converted to the USKB; (b) TOEFL iBT Writing scores appear to be significantly associated with the types of keyboard and overlay in only 10 countries, with trivial or small score differences associated with keyboard type. These findings suggest that the current practices related to keyboard type appear to have no or little practical effect on examinees’ TOEFL iBT Writing scores.