INVESTIGATING ITEM ORDER EFFECT ON PSYCHOMETRIC PROPERTIES OF A SELF-REPORT SCALE

Abstract

In this study, the item order effect was investigated using the self-efficacy perception scale for computational thinking. 946 participants consisting of 8th grade students participated in the research. Participants were first administered the original form of the scale, in which the dimensions were sequential and each item was presented in the relevant sub-dimension. One month later, a second form in which the order of the items was completely random was administered to the same group. Analyzes revealed no significant difference in the group's total mean score between the two forms. Additionally, the study showed the extent to each individual participants' total and factor scores varied between two forms, and the difference was negligible. Another remarkable finding is that sequentially arranging items representing the same dimension contributes positively to the internal consistency reliability of the scale. Confirmatory factor analyzes were performed for both forms, revealing good model fit with similar index values; This shows that randomizing the item order does not disrupt the structure of the scale. In addition, factor loading differences between the models were also examined. Finally, the analysis results showed that the two models achieved strict level of measurement invariance that is often difficult to achieve in practice. Contextual effects of items can be observed more in attitude scales. To investigate this effect in future studies, instead of creating random forms, forms in which the order of the items is consciously manipulated for a specific purpose can be used. It is also suggested that the item order effect can be examined in the context of many demographic characteristics and different item types.