Analyzing Likert scale surveys with Rasch models

Abstract

Likert-scale surveys have been used in applied linguistics to make many research claims, and thus it is important to validate our score interpretations. The present study aims to demonstrate the usefulness of the Rasch partial credit model (PCM) and the Rasch rating scale model (RSM) for score interpretations of Likert-scale surveys given these models’ limited use in applied linguistics. The study used publicly available data shared by Weng (2020), where 182 students in foreign language programs at an American college responded to an eight-item, five-point Likert-scale survey intended to measure foreign language anxiety (FLA). The RSM and the PCM flagged two items that are potentially problematic due to item negation, item reversal, and double-barreled wording. The two models also suggested that the strongly disagree and strongly agree options help distinguish respondents. Content analysis also indicated that the undecided option may represent FLA only for one item, questioning the treatment of 3 as higher than 1 and 2 or as lower than 4 and 5 on an interval scale. These findings suggest that raw scores need to be interpreted cautiously, highlighting the great potential of the RSM and the PCM to facilitate our score interpretations of Likert-scale surveys.