The reliability and validity of a scale through the comparison between the traditional method and the Rasch model

Abstract

The present study addressed the issue of reliability and validity of scale through the comparison between the traditional method and the Rasch model, which is seen by statisticians as the best method to psychometrically validate the scale and test its properties. Researchers have demonstrated the failure of traditional statistical methods to take into account the characteristics of the items and people when testing the reliability of the scale as well as respondents. They usually turn to alpha Cronbanch to examine the internal consistency of the scale without taking into consideration that alpha Cronbanch has been affected by external factors such as the length of the scale. However, the Rasch model that is relatively considered to be modern statistics, is not affected by external factors especially the length of the scale, even a short scale might be more reliable than the long one. Moreover, the Rasch model can also be used to investigate various types of validities such as content validity, construct validity, and criterion validity. Interestingly, the Rasch model is also a powerful statistics tool used to determine the contribution of the items and people in the total reliability of the scale. For example, the variable map is used to examine the extent to which the items adequately target the respondents taking into account the difficulty of items and ability of the subjects. Nevertheless, the traditional methods normally calculate the uniqueness of the scale by focusing on the sum of squares and these methods do not offer standard errors for each item to determine the accuracy of the measurement. Hence, the researchers suggested in this paper that favorability of the Rasch model in testing scales’ reliability and validity was recommended compared to conventional statistical methods.