Process differences as a function of test modifications: Construct validity of Raven's advanced progressive matrices under standard, abbreviated and/or speeded conditions – A meta-analysis

Abstract

Historically, there has been substantial disagreement about the importance of speed vs. level in determining individual differences in intelligence – a disagreement that persists across various different modern assessment measures of intellectual abilities. The current investigation considers whether changes to the administration constraints (time limitations or speededness, and total test length) of the Raven's Advanced Progressive Matrices test – which has been identified as a measure highly saturated with general intelligence – results in differences to the underlying ability determinants of test performance. A review of empirical studies was conducted, where versions of Raven's Advanced Progressive Matrices Tests were administered under various time constraints and item lengths. Meta-analytic techniques were used to determine whether introducing speed constraints or shortening the length of the test changes the construct validity of the tests (as indicated by differences in convergent and discriminant correlations with other ability traits). The meta-analysis combined results from 142 studies composed of a total of 26,848 participants. Substantial differences were found for correlations of Raven's Advanced Progressive Matrices and Spatial Visualization (as large as ρ̂ = 0.26), Memory (as large as ρ̂ = 0.08), and Perceptual Speed (as large as ρ̂ = 0.34) abilities under speeded conditions and shorter test lengths. Examinees may draw on different strategies for test performance, that in turn, draw on different combinations of abilities, when the test is abbreviated or significant time constraints are introduced. Implications for using this test under different conditions are discussed.