The Effect of Proportion of Mixed-Format Scoring: Mixed-Format Achievement Tests

Abstract

Mixed-format Tests combine Multiple Choice questions test (MC) with Dichotomous Item Response (DS) and Construct Response test (CR) with Polytomous Item Response (PS). Combining two types of items on one assessment enhances both the reliability and validity of the assessment, thereby enhancing the strength of multiple-choice test and Construct response test. The problems encountered in the design are also evident in the proportion of tests scoring Dichotomous and Polytomous Item Response. It allows the use of a combination of high efficiency test. The objective of this research was to study and compare the efficiency of Mixed-format tests when the proportion of tests scoring Dichotomous and Polytomous Item Response are different (20: 80 50:50 and 80:20) The efficiency evaluation of the Mixed-format tests considers the standard error of estimation (SE(θ) index and BIAS index. The MANOVA was analysed to compare the mean of SE(θ) index and BIAS index. Monte Carlo method was used for data simulation. Dichotomous and Polytomous Item Response Models, i.e. Three- parameter Logistic Model and Generalized Partial Credit Model (GPCM) are used for marking. The research findings showed proportion of the tests that used Dichotomous and Polytomous Item Response affected SE(θ)and BIAS at 0.5 significant level. Moreover, proportion of the tests that used different Dichotomous and Polytomous Item Response affected SE(θ) and BIAS at 0.5 significant level. In addition, SE(θ) at ability level of -3.0 0.0 and 3.0 as well as BIAS would have the least value at the proportion of tests that used Dichotomous and Polytomous Item Response of 20:80, 50:50 and 80:20 respectively.