On the use of rotated context questionnaires in conjunction with multilevel item response models

Raymond J Adams,Petra Lietz,Alla Berezner

doi:10.1186/2196-0739-1-5

Abstract

While rotated test booklets have been employed in large-scale assessments to increase the content coverage of the assessments, rotation has not yet been applied to the context questionnaires administered to respondents. This paper describes the development of a methodology that uses rotated context questionnaires in conjunction with multilevel item response models and plausible values. In order to examine the impact of this methodology on the continuity of the results, PISA 2006 data for nine heterogeneous countries were rescaled after having been restructured to simulate the outcomes of the use of different rotated context questionnaire designs. Results revealed negligible differences when means, standard deviations, percentiles, and correlations were estimated using plausible values drawn with multilevel item response models that adopted different approaches to questionnaire rotation. The results of the analyses support the use of rotated contextual questionnaires for respondents in order to extend the methodology currently used in large-scale sample surveys.

Highlights

While rotated test booklets have been employed in large-scale assessments to increase the content coverage of the assessments, rotation has not yet been applied to the context questionnaires administered to respondents
The ultimate aim of the survey is to examine the distribution of the latent variable in the target population and to make inferences concerning the relationships between latent variables and other variables in the target population
We explore the possibility of administering rotated context questionnaires to respondents in order to expand the coverage of contextual variables in sample surveys that employ multilevel item response theory scaling models

Summary

Introduction

While rotated test booklets have been employed in large-scale assessments to increase the content coverage of the assessments, rotation has not yet been applied to the context questionnaires administered to respondents. A common goal of sample surveys is to measure a latent variable proficiency, an aptitude, an attitude, or the like and relate that latent variable to other characteristics of the respondents. In an educational context, the relationship that is examined might be the correlation between the latent variable and another characteristic, such as years of schooling, or it might be between-group differences in mean scores for a latent variable. In psychological and educational research, the presence of substantial measurement error resulted in the development of linear structural relation (or LISREL) models (see, for example, Jöreskog & Sörbom 1984; Muthén 2002) and latent regression ( known as multilevel item response theory) models (Adams, Wu, & Carstensen, 2007; Fox & Glas 2002)

Objectives

Methods

Results

Conclusion