Multidimensional or essentially unidimensional? A multi-faceted factor-analytic approach for assessing the dimensionality of tests and items.

Abstract

Factor-analysis based dimensional assessment of psychometric measures is a key step in the development of tests. However, current practices for deciding between a multiple-correlated or essentially unidimensional solution are clearly improvable. A series of recent studies are reviewed, and an approach is proposed that combines multiple sources of information, which is expected to be used to make an informed judgement about the most appropriate dimensionality for the measure being studied. It uses both internal and external sources of information, and focuses on the properties of the scores derived from each of the solutions compared. The proposal is applied to a re-analysis of a measure of symptoms of psychological distress. The results show that a clear and informed judgement about the most appropriate dimensionality of the measure in the target population can be obtained. The proposal is useful and can be put into practice by using user-friendly, non-commercial software. We hope that this availability will result in good practice in the future.