Severity and Leniency Effect in Assessing Teaching Practicum: Analysis using Multi Facet Rasch Measurement (MFRM)

Abstract

Practical teaching assessment is a performance assessment that can be exposed to threat regarding the validity and reliability as the assessment is made subjectively. If it is not measured correctly, it can give effect to a negative score. The purpose of this study is to analyse the severity and leniency effects between raters through a variable map in Multi Facet Rasch Measurement (MFRM) analysis. Instrument of study is an e-PLM assessment rubric that is assessed based on five assessment criteria which are planning, implementation, personality, portfolio and reflection. This study is done quantitatively. Data were collected from 170 of teachers candidate that undergoing teaching practical in Malaysia peninsula and 194 raters that gave the rate towards all the teachers candidate. Raters consists of the lecturers from UTHM and teacher’s counselor from Vocational College who are sampled by using purposive sampling technique. Data was analyzed using analysis software Multi Facet Rasch Measurement (MFRM) Version 3.82.2. In the study, it was reached to the results that the rater’s severity holds an unexpected effect towards group level and individual level in assessment based on the logit value, infit MNSQ, outfit MNSQ, Model S. E and t-value. Based on results analysis, three rater are found to be more severe and inconsistent in assessing students. While, eight raters are found to be more lenient in making the assessment. It can be concluded that these raters was inconsistent in rating student performance. From the findings, the result shows that there are severity/leniency effects at the individual level while severity/leniency effects does not exist at the group level. However, to summarize this study gives a huge impact on validity and reliability towards performance assessment. Besides that, this study also can assist rater in the assessment field, assessment practice as well as lead towards the valid and quality assessment.