Validated tests for language research with university students whose native language is English: Tests of vocabulary, general knowledge, author recognition, and reading comprehension.

Abstract

We present five studies aimed at developing an L1 vocabulary test for English-speaking university students. Such a test is useful as an indicator of crystallized intelligence and because vocabulary size correlates well with reading comprehension. In the first study, we tested 100 written words with four answer alternatives, based on Nation's Vocabulary Size Test. Analysis suggested two factors, which we interpreted as the possible existence of two types of difficult words: unknown words for general knowledge and unknown words for specialized knowledge. In Study 2, we attempted to develop a vocabulary test for each type of word, and these tests were then validated in Study 3. Since the test for general words proved too easy for the target population, we improved it in a fourth study by creating and testing more difficult items. Finally, a fifth study was conducted to validate the new test. Unexpectedly, Study 5 found a high correlation (r = .82) between the general knowledge vocabulary test and the specialized knowledge vocabulary test, suggesting that they measure the same latent factor, contrary to our initial assumption. Both tests have high reliability (r > .85) and correlate well (r > .4) with general knowledge, author recognition, and reading comprehension. In addition, a collection of other language tests was used and improved to verify the validity of the vocabulary tests. An exploratory factor analysis of all tests identified three factors (text comprehension, crystallized intelligence, and reading speed), with the vocabulary tests loading on the factor crystallized intelligence, which in turn correlates with reading comprehension. Structural equation modeling confirmed the interpretation.