Test Biases of a Listening Comprehension Test Administered to Japanese and Korean High School Students : An Analysis Based on the Classical Test Theory and Item Response Theory

Abstract

In English language teaching in junior and senior high schools, development of listening comprehension tests focusing on oral communication abilities is needed due to the shift of the course of study toward a more communication-oriented approach (Monbusho 1989). Our previous studies (Kinoshita et al. 1996, 1998a, 1998b) reported the formation of a listening comprehension test focusing on oral communication abilities and presented an analysis of the test which was administered to Japanese and Korean high school students. The results of our previous analysis (Kinoshita et al. 1998a, 1998b) indicated that there were no significant differences between the scores of Japanese and Korean test takers, though some differences were found to be statistically significant in the performance of certain aspects of listening skills. Our previous studies, however, could not identify clearly why some differences happened because several conditions of the test takers differed. This study deals with only the test data taken from the test takers under the same condition and reanalyzes the data based on the Classical Test Theory and Item Response Theory. As a result, we find that some test items are biased by cultural differences between Japan and Korea and some differences are due to the differences of the instruction itself. To test listening comprehension abilities focusing on oral communication, it is important to take account of the cultural differences among the test takers at a high school level. Finally, our results suggest that the way to teach listening in Japan needs to be revised because Japanese test takers show poorer performance in test items focusing on everyday situations.