A Mokken Scale Analysis of the Last Series of the Standard Progressive Matrices (SPM-LS).

Nils Myszkowski

doi:10.3390/jintelligence8020022

Abstract

Raven’s Standard Progressive Matrices (Raven 1941) is a widely used 60-item long measure of general mental ability. It was recently suggested that, for situations where taking this test is too time consuming, a shorter version, comprised of only the last series of the Standard Progressive Matrices (Myszkowski and Storme 2018) could be used, while preserving satisfactory psychometric properties (Garcia-Garzon et al. 2019; Myszkowski and Storme 2018). In this study, I argue, however, that some psychometric properties have been left aside by previous investigations. As part of this special issue on the reinvestigation of Myszkowski and Storme’s dataset, I propose to use the non-parametric Item Response Theory framework of Mokken Scale Analysis (Mokken 1971, 1997) and its current developments (Sijtsma and van der Ark 2017) to shed new light on the SPM-LS. Extending previous findings, this investigation indicated that the SPM-LS had satisfactory scalability (), local independence and reliability (, ). Further, all item response functions were monotonically increasing, and there was overall evidence for invariant item ordering (), supporting the Double Monotonicity Model (Mokken 1997). Item 1, however, appeared problematic in most analyses. I discuss the implications of these results, notably regarding whether to discard item 1, whether the SPM-LS sum scores can confidently be used to order persons, and whether the invariant item ordering of the SPM-LS allows to use a stopping rule to further shorten test administration.

Highlights

The general factor of intelligence (g) is central in the prediction of several outcomes, such as job performance (Ree and Earles 1992; Salgado et al 2003) and academic achievement (Rohde and Thompson 2007).Its accurate measurement is crucial in multiple contexts, including personnel selection, vocational guidance or academic research in individual differences
I discuss the implications of these results, notably regarding whether to discard item 1, whether the Standard Progressive Matrices (SPM)-LS sum scores can confidently be used to order persons, and whether the invariant item ordering of the SPM-LS allows to use a stopping rule to further shorten test administration
While the SPM-LS has already been investigated using a variety of methods in this very dataset—including parametric IRT, Bayesian IRT, factor analysis, and exploratory graph analysis (Myszkowski and Storme 2018; Garcia-Garzon et al 2019; Bürkner 2020)—the current study proposes the first investigation of this instrument using non-parametric IRT, and Mokken Scale

Summary

Introduction

Its accurate measurement is crucial in multiple contexts, including personnel selection, vocational guidance or academic research in individual differences. Because of practical constraints, it is desirable in many contexts to reduce test length as much as possible, while maintaining acceptable accuracy. Raven’s Standard Progressive Matrices (SPM) (Raven 1941) and Advanced Progressive Matrices (APM) (Raven et al 1962) are widely used—though criticized (Gignac 2015)—instruments to measure g. Both these tests remain rather long in untimed conditions, with some participants sometimes taking more than 40 min to respond them (Hamel and Schmittmann 2006). Several solutions have been proposed to further reduce test administration time, such as constraining time (Hamel and Schmittmann 2006) and using short versions (Bors and Stokes 1998)

Methods

Results

Discussion

Conclusion