Abstract
The article presents a study of a CEFR B2-level reading subtest that is part of the Slovenian national secondary school leaving examination in English as a foreign language, and compares the test-taker actual performance (objective difficulty) with the test-taker and expert perceptions of item difficulty (subjective difficulty). The study also analyses the test-takers’ comments on item difficulty obtained from a while-reading questionnaire. The results are discussed in the framework of the existing research in the fields of (the assessment of) reading comprehension, and are addressed with regard to their implications for item-writing, FL teaching and curriculum development.
Highlights
Following the well-established distinction between the objective and subjective difficulty (Fulmer and Tulis 2013), the present study aims at determining possible correlations and interdependencies between these two types of difficulty, with special attention being paid to their importance for item-writers, test, policy- and curriculumdevelopers as well as CEFR1-alignment experts
In our context, most of the language experts participating in the CEFR alignment project are item-writers for the national examinations, and second we want to address the question of experts and their reported weak ability to predict the item/task difficulty (Alderson and Lukmani 1989; Sydorenko 2011)
Following Freedle and Kostin’s (1999) classification of items, we identified detail explicit (D-E) items (12 items), detail implicit (D-I) items (2 items), gist explicit (G-E) items (2 items), gist implicit (G-I) items (2 items), and items targeting at textual organisation/structure (O-S) (10 items)
Summary
For the purposes of the investigation, the GM reading subtest has been administered to a group of testtakers together with a while-reading questionnaire, in which the test-takers have commented on their perception of the item/task difficulty. In order to determine to what extent the objective difficulty correlates with the subjective difficulty, the study compares (i) the psychometric measurements of the reading subtest (objective difficulty) with (ii) the answers from the while-reading questionnaire as well as with the judgments of the language experts that have aligned the GM examination with the CEFR (subjective difficulty). In our context, most of the language experts participating in the CEFR alignment project are item-writers for the national examinations, and second we want to address the question of experts and their reported weak ability to predict the item/task difficulty (Alderson and Lukmani 1989; Sydorenko 2011). We strongly believe that apart from theoretical implications, the results of our investigation will have practical value especially in educational environments where the test-provider does not follow all the standardised test-design procedures as clac 67/2016, 318-342
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
