Using Alignment Index and Polytomous Item Response Theory on Statistics Essay Test

Abstract

Purpose: Essay test in mathematics, both in the form of restricted-response and extended-response, generally consist of polytomous scored items. However, the essay test used by teachers in Indonesia has not been fully supported by sufficient quality evidence. There have been many studies focusing on the development of the essay test, but not many of them have applied the use of relevant measurement theory for the polytomous data. The evidence of content validity also has not been supported by its alignment with the curriculum. This study used alignment ind ex to prove the content validity and IRT polytomous GPCM to determin e the characteristics of test items in order to produce a n essay test that could accurately measure the achievement of students on statistical materials. Method : Procedures of this study: (1) preparation of preliminary test, (2) trials, (3) interpretation. Trial was conducted involving 688 Junior High School students in Yogyakarta, Indonesia. Result s: The content validity of the test wa s good, supported by V Aiken index of 0.88–1.00 and Porter alignment index of 0.93. The test items ha d good construct validity . Test reliability wa s categorized as good with the Construct Reliability coefficient of 0.88 and the Alpha coefficient of 0.78. Judging from its characteristics, all test items were categorized as good . Implications for Research and Practice : The use of the alignment index contribution to the verifica tion of content validity of essay test and the use of the IRT polytomous GPCM may provide reference for the use of appropriate measurement theory to determine the item characteristics of essay test.