Is Writing Performance Related to Keyboard Type? An Investigation From Examinees’ Perspectives on the TOEFL iBT

Abstract

ABSTRACTTo investigate whether the type of keyboard used in exams introduces any construct-irrelevant variance to the TOEFL iBT Writing scores, we surveyed 17,040 TOEFL iBT examinees from 24 countries on their keyboard-related perceptions and preferences and analyzed the survey responses together with their test scores. Results suggest that controlling for their TOELF iBT Reading, Listening, and Speaking scores, examinees’ Writing scores were statistically significantly associated with examinees’ prior awareness of the TOEFL iBT keyboard policy, the type of keyboard they used during the test, or examines’ perceptions of the type of keyboard used. However, the score differences associated with these factors were negligible, supporting the argument that the current practice of mandatorily using the U.S. standard keyboard poses limited threat to the validity of test scores. Survey results also revealed that although most examinees considered the keyboard used during the test convenient and efficient, two-fifths of them reportedly preferred a more familiar local keyboard, which suggests room for improvement in the keyboard-related test-taking experience.