Determining English Language Lecturers’ Quality of Marking in Continuous Assessment through Rasch Analysis

Abstract

English language lecturers at matriculation colleges are generally equipped with assessment criteria for marking students’ written assessment. However, these criteria are normally susceptible to lecturers’ interpretation and understanding, which threatens quality marking. Therefore, this study aimed to determine the severity and consistency of English language lecturers’ marking of English academic writing (EAW) in continuous assessment. The participants were five English language lecturers and 50 matriculation students. Each lecturer selected ten EAWs randomly from 318 matriculation students. The five-part EAW was marked first by the class’s lecturer and later, it was marked by pre-assigned peer moderators who did not teach the students. The total data set collected was 250 (5 lecturers x 10 EAWs x 5 parts of EAW). The data were analyzed with Many-Facets Rasch Measurement (MFRM) application. Semi-structured interviews were conducted with both lecturers and students for triangulation purposes. Findings revealed that four out of five lecturers were lenient in marking but the marking was found to be internally consistent with infit and outfit mean squares for each lecturer ranged between 0.5 and 1.5. From interview responses analyzed, students perceived their lecturers as fair but strict in awarding marks. These responses were consistent with most lecturers’ responses on their strict adherence to assessment criteria. Discussion of findings is centered on the issue of severity and consistency of the assessors. This study could offer a practical solution in providing evidence for quality marking of written assessment and, consequently, aid in developing remedial measures for misfit assessors in educational institutions.