Application of the Bookmark method: setting standard for the ninth-grade mathematics achievement test in China

Abstract

The purpose of this paper is to apply the Bookmark method to the standard setting. Based on the Rasch Model in item response theory, a ninth-grade mathematics achievement test in china has been taken as an example of the standard setting, and 2 cut scores have been established to distinguish students into different performance levels eventually, namely basic and proficient cut scores. In addition, based on the use of generalizability theory, the standard error of the cut scores and the practical standard error are used as indicators to explore the effect that panelists and the standard setting rounds have made on the precision of Bookmark standard setting results through a mixed design of (p: g) × r. Result shows that the cut scores of basic and proficient were respectively 52.25 and 67.53. Besides, increasing the number of panelists in the group or standard setting rounds will reduce the standard error of the cut scores and the practical standard error. In addition, practical standard error is a necessary reference index when applying generalizability theory to analyze the cut scores established by Bookmark method, while the standard error of cut scores also has a great reference value.