Characteristics of Neural Test Theory: Comparison With Classical Test Theory and Rasch Modeling

Abstract

The neural test theory (NTT) 'uses the mechanism of a self-organizing map' or generative topographic mapping and 'assumes the latent scale is ordinal' (Shojima, 2008c). The current study aims to reveal the characteristics of the NTT by applying it to the analysis of a placement test at a university. We compared the NTT results with those obtained using the Classical Test Theory (CTT) and Rasch modeling (RM). The participants comprised 147 Japanese learners of English, whose major subject was international studies or management studies. They took a 90-item multiple-choice vocabulary test. We obtained the test scores using the CTT (percentages correct), RM (latent ability estimates), and NTT (latent rank estimates), and classified the students into three or five groups with different proficiency levels based on the scores derived from the CTT and RM. After a detailed analysis, we ascertained three findings. First, our analysis revealed that the three types of scores and the two types of groups were highly correlated. This suggests that similar results can be obtained by using any one of the three test theories. Second, the maximum number of groups into which we could divide the students was the same (i.e., three) according to the separation index in the RM and the test model-fit indices in the NTT. Third, we compared the item difficulty and discrimination obtained from these three theories and showed that the results of the item difficulty using the CTT, RM, and NTT were highly correlated; similar results were observed for item discrimination computed using the CTT and NTT. Overall, the NTT results (i.e., the test-takers' latent ranks, the maximum number of groups, item difficulty, and item discrimination) are similar to those obtained using the CTT and RM. Furthermore, the NTT is advantageous in computing ordinal ranks based on test-takers' test response patterns with a relatively small sample size and in presenting more information on item monotonicity. Thus, the present study provides evidence for the effectiveness of the NTT in analyzing in language testing data, especially when only ordinal scale results are required.