Achievement Test Item Analysis: A Comparison of Traditional and Modern Methods

Abstract

One of the great advantages of the modern approach to item and test analysis, namely, the Item Response Theory (IRT) over the traditional Classical Test Theory (CTT) is that item statistics such as the difficulty index and the discrimination index are not sample dependent. Very few research findings using IRT models have been published locally. Hoffman is one of the small group of researchers who have implemented IRT models in South Africa. The purpose of the present study was to compare CTT and IRT item and test analysis. Data from the multiple choice part of the 1988 examination paper on research methodology for 538 undergraduate UNISA Industrial Psychology students were used. Results showed that corresponding interpretations of item statistics (CTT) and item parameters (IRT) was possible under the two approaches. More or less the same conclusions could also be drawn with regard to evaluation of test statistics. The near-perfect correlation between test scores (CTT) and ability estimates (IRT) for the one-parameter model indicates that a great similarity exists between them.