The impact of frequency rating scale formats on the measurement of latent variables in web surveys - an experimental investigation using a measure of affectivity as an example

Abstract

The effects of verbal and/or numerical labeling and number of categories on the measurement of latent variables in web surveys are addressed. Data were collected online in a quota sample of the German adult population (N = 741). A randomized 2x2x2 experimental design was applied, with variation of the number of categories, as well as of verbal and numerical labeling, using an abbreviated version of the Positive and Negative Affect Schedule (PANAS). Experimental manipulation of the rating scale formats resulted in an effect on measurement model testing and reliability, as well as on factorial and convergent validity. In addition, measurement invariance between several rating scale formats was limited. With the five category end verbalized and fully labeled seven category formats, acceptable results for all measurement quality metrics could be obtained.