Item Difficulty Predictors of a Multiple-choice Reading Test

Abstract

The researchers’ experiential knowledge demonstrates that the task of predicting and controlling the difficulty level of the multiple-choice items of the College Scholastic Ability Test (CSAT) for English is substantially left to the subjective judgment of experienced item writers. The present study accordingly recognizes a need to identify item difficulty predictors and build an item difficulty prediction model to handle this pertinent issue. While taking separate interest in constructing a model for the multiplechoice reading subset of the CSAT, the study was conducted by identifying item difficulty predictor variables from previous research, and by validating the candidate predictors via questionnaires by highly experienced teacher-raters when asked to analyze reading items from the English reading subset of the preliminary CSAT (i.e., yun-hap-hak-ryuk-pyung-ga) administered in March 2009. Using multiple regression technique and maximum likelihood estimation, an item difficulty prediction model was generated. In order to check validity and applicability of the prediction model, the hypothetical model was finally tested on a subsequent version of the test administered in September 2009. This type of model building is expected to guide test developers design an item pool in accordance with special needs, such as to construct multiple test forms, which have similar mean difficulties.