Evaluation of internal validity using modern test theory: Application to word association.

Abstract

Word association tests (WATs) have been widely used to examine associative/semantic memory structures and shown to be relevant to behavior and its underpinnings. Despite successful applications of WATs in diverse research areas, few studies have examined psychometric properties of these tests or other open-ended cognitive tests of common use. Modern test theory models, such as item response theory (IRT) models, are well suited to evaluate interpretations of this class of test. In this evaluation, unidimensional IRT models were fitted to the data on the WAT designed to capture associative memory relevant to an important applied issue: casual sex in a sample of 1,138 adult drug offenders. Using association instructions, participants were instructed to generate the first behavior or action that came to mind in response to cues (e.g., "hotel/motel") that might elicit casual sex-related responses. Results indicate a multitude of evidence for the internal validity of WAT score interpretations. All WAT items measured a single latent trait of casual sex-related associative memory, strongly related to the latent trait, and were invariant across gender, ethnicity, age groups, and sex partner profiles. The WAT was highly informative at average-to-high levels of the latent trait and also associated with risky sex behavior, demonstrating the usefulness of this class of test. The study illustrates the utility of the assessments in this at-risk population as well as the benefits of application of the modern test theory models in the evaluation of internal validity of open-ended cognitive test score interpretation. (PsycINFO Database Record