OPTIMISING ASSESSMENT SYSTEM IN THE ESP COURSE THROUGH THE USE of THE METHODS OF DIFFERENTIAL ITEM FUNCTIONING AND DIFFERENTIAL TEST FUNCTIONING IN FINAL TEST DESIGN

Abstract

The purpose of the research was to examine how the use of the methods of Differential item functioning and Differential test functioning contribute to the quality of the final assessment (FT-ESP) in the English for Specific Purposes course delivered to the graduate students at tertiary institutions. The study relies on two interventions intended to identify the correlation between the test design and the academic performance of the students in the ESP course through using Pearson’s correlation coefficient of the answered versus the unanswered questions. The first intervention test was similarly structured as the one for the second intervention and consisted of the same number of items. In the first intervention, a regular final ESP test was administered. In the second intervention, the originally designed test, which validity and reliability was analyzed using the methods of DIF and DTF, was performed. The test included three sub-domains such as: reading comprehension (15 items), structure (15 items), and compositional analysis (15 items). It has been found that the use of the methods of DIF and DTF boosts the quality of the assessment system in the ESP course delivered to the graduate students at tertiary institutions. It is advisable that the first step in DIF analyses be related to the use of statistical methods to detect the DIF items. It is also advisable to examine the effects of other potential factors on DIF such as item order and mother tongue effects along with unintended content specific factors to explain DIF effect in the context of language testing. The findings also imply that neither of methods addresses the issue of measurement bias, which might occur in tests, because it is complicated and cannot be addressed adequately using simple statistical or classical test theory methods. Further studies are needed to identify the ways of improving the assessment of speaking skills of the graduates of the tertiary institutions.