Developing an Essentially Unidimensional Test With Cognitively Designed Items

Abstract

The purpose of this study was to demonstrate how cognitive and measurement principles can be integrated to create an essentially unidimensional test. Two studies were conducted. In Study 1, test questions were created by using the feature integration theory of attention to develop a cognitive model of performance and then manipulating complexity factors within the model. It was hypothesized that the complexity factors predict item difficulty. Results indicated that some complexity factors predicted difficulty in a relatively small sample. In Study 2, items developed using the cognitive model were integrated with items measuring another factor to create a multidimensional test of spatial reasoning. Results were replicated in Study 2 with a sample of 460 participants. The test met the assumption of essential unidimensionality according to DIMTEST, was moderately correlated (r = .64) with the Bennett Mechanical Comprehension Test, and showed little evidence of differential item functioning. Implications are discussed.