THE FEASIBILITY OF USING A TEST GENERATOR TO AMELIORATE LEARNING EFFECTS IN REPEAT COGNITIVE TESTING

Abstract

The accurate and valid measurement of cognitive change is important for providing dementia trials outcomes, describing cognitive trajectory in population studies, and for detecting early impairment. Repeating traditional tests has shown clear learning effects, in both laboratory (Salthouse et al, 2004) and epidemiological studies (Gallacher et al, 2008). A major determinant of learning effect is familiarity with the test items. Using fluid intelligence as the cognitive domain, and Raven's Standard Progressive Matrices (SPM) as the test paradigm, a test generator that enabled item familiarity to be manipulated is demonstrated as an option for use in pharmacologic and epidemiologic studies. Corvus, a Raven's-like test generator was designed and built. Item familiarity can be manipulated using varying similarity between element attributes, layouts, rules and answer sets. To investigate the impact of item familiarity data were collected from 25 individuals who were randomised to complete either tests generated by Corvus or SPM on 5 occasions over a 14 day period. There were two indices of performance, test duration and test score. Test duration reduced over the five occasions for both Corvus (Figure 1) and SPM (Figure 2). However SPM showed a significantly greater reduction in test duration. Additionally participants taking Corvus sped up at a relatively constant rate throughout the test, while participants taking SPM sped up to a greater degree on harder questions and begin to approach a constant time per item regardless of difficulty. For test score there was only marginal change for both tests. The use of item generators to produce parallel test versions has advantages over the use of parallel test items. Data presented here shows that a test generator (Corvus) can be used to assess fluid intelligence and ameliorates item familiarity effects.