The Development of Diagnostic Test Using the Attribute Hierarchy Method

Abstract

Nowadays, to investigate students’ potentiality, any exam has to contain a lot of questions. Therefore, using attribution hierarchy model will be able to reduce the number of questions. This study addresses the following research question: Which attributes are needed to be included in the diagnostic test and How much is the level of quality of the diagnostic test? The purpose of this study was to develop a diagnostic test and to quantify its quality. The study will concentrate in students’ mathematical problem-solving skill. Moreover, the created diagnostic test will limit to the skill of solving adding and subtracting fractions questions only. The diagnostic test will be developed based on the attribute hierarchy model. To quantify the level of ability of students, the Bayes’ Theorem was employed to determine the diagnostic score. Moreover, to quantifying the quality of the diagnostic test, 2-parameter item response model, content validity, Hoyt's reliability, and inter-rater reliability are used. The diagnostic test contained only 17 questions which are relatively low comparing to the current test. It is found that the difficulty parameter is between -2.14 and 1.06, the item discrimination parameter is 0.77 – 3.07, the content validity is 1.00, Hoyt's reliability is 0.84 and the inter-rater reliability is 0.90.