영어듣기시험 문항 타당도에 영향을 주는 요인 분석

Abstract

The aim of this study is to investigate the characteristics of the nationwide standardized listening test items for middle school students in Korea. The test items are evaluated in terms of item difficulty, item discrimination power, fit statistics, and attractiveness of choices. Based on evidence from the quantitative analyses, the content of individual items was analyzed. In this study, responses from 271 students on a listening test of 20 items were collected and analyzed using Rasch measurement model and 3-parameter IRT model. The findings are as follows. First, reliability is .85 indicating that the test is reliable, but there were some misfit, overfit, and low point-biserial test items found including a variety of difficulty levels and discrimination index. Second, several items which show unexpected responses were selected critically to review the listening texts, question types, and choices. Finally, some suggestions and implications can be discussed for the development of listening tests. In order to enhance the test validity and reliability, constructs to measure listening ability should be refined and elaborated to fit the question types. Listening test length also should be designed by considering the listening process as well as efficiency of distracters.