Multidimensional Item Responses: Multimethod-Multitrait Perspectives

Abstract

In this paper we discuss complexities of measurement that can arise in a multidimensional situation. All of the complexities that can occur in a unidimensional situation, such as polytomous response formats, item dependence effects, and the modeling of rater effects such as harshness and variability, can occur, with a correspondingly greater degree of complexity, in the multidimensional case also. However, we will eschew these, and concentrate on issues that arise due to the inherent multidimensionality of the situation. First, we discuss the motivations for multidimensional measurement models, and illustrate them in the context of a state-wide science assessment involving both multiple choice items and performance tasks. We then describe the multidimensional measurement model (NMRCML). This multidimensional model is then applied to the science assessment data set to illustrate two issues that arise in multidimensional measurement. The first issue is the question of whether one should (or perhaps, can) design items that relate to multiple dimensions. The second issue arises when there is more than one form of multidimensionality present in the item design: should one use just one of these dimensionalities, or some, or all? We conclude by discussing further issues yet to be addressed in the area of multidimensional measurement.