Abstract
Task difficulty is a critical issue affecting test developers. Controlling or balancing the item difficulty of an assessment improves its validity and discrimination. Test developers construct tests from the cognitive perspective, by making the test constructing process more scientific and efficient; thus, the scores obtained more precisely represent the proficiency level. In this paper, a framework of cognitive factors related to English listening comprehension (LC) tests is proposed. Data used were derived from the After School Alternative Program (ASAP) English test item pool. A total of 150 items were analyzed, and item difficulty was concurrently calibrated according to three-parameter-logistic item response theory from the responses of 1,459 fifth- to seventh-grade participants. Components affecting item difficulty were proposed and discussed with regard to the cognitive perspective. The 53.5% of variance in item difficulty of LC can be explained by the cognitive predicting model. This result is expected to make the task constructing procedure more organized and to ensure the task is of the desired difficulty, thus enabling the intended ability to be effectively measured. In addition, the ASAP English test aims to assist low-achieving students in remedial instruction; the cognitive factors and study results provide a reference for developing teaching materials of English LC remedial instruction and for constructing LC test items.
Highlights
We investigated the cognitive factors affecting the task difficulty of the After School Alternative Program English Listening comprehension (LC) test (ASAP-ENGLCT)
This study investigated the cognitive factors affecting the item difficulty of LC based on the ASAP-ENGLCT
A multiple regression model was used for investigating the correlation between item difficulty and the cognitive factors, Table 6 shows that all the nine cognitive factors were significantly correlated with item difficulty
Summary
Listening is a cognitive process and, in reality, its context is not visible to the listener. Top-down process uses background knowledge and common sense for comprehending aural stimuli This process involves “attention to a continuous stream of speech, which is not under the timing control of the listeners” [3]; this process imposes a heavy cognitive load on listeners. Cognitive tasks (including test items) are considered to require multiple processing stages, strategies, and knowledge stored. Both tasks and people vary on the processing components. Knowledge about cognitive structures can help test developers, psychologists, and educators to construct test items with desirable measurement and cognitive characteristics, operate constructs, and satisfactorily understand the cognitive processes of thinking and performance. Test constructers can use the factors obtained here for developing tests and manipulating item difficulty to ensure satisfactory construct validity in the test
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call