The Use of Letter Prompts in Vocabulary Size Tests

Abstract

This study examines the score inflation of vocabulary size tests in conjunction with the use of letters in the item response component. The letter prompt is used to constrain test taker responses to target vocabulary, but may also have the unintended consequence of assisting test takers in answering test items. In such cases, test scores would be a reflection not only of a test taker’s ability, but also of the use of letter prompts. To investigate this issue, receptive and productive vocabulary size tests with and without letter prompts were administered to 67 university students. The results support the view that the use of letter prompts results in higher test scores as compared to tests that do not use such prompts, and highlight the heightened efficacy of the letter prompts for less well-known words. However, an error analysis confirmed the threat of test takers answering with synonyms of the target answer words when not constrained by letter prompts, thus presenting test designers with conflicting validity concerns: score inflation and non-target responses.