Abstract
The present study tries to investigate the fairness of an English reading comprehension test employing Kunnan’s (2004) test fairness framework (TFF) as the most comprehensive model available for test fairness. The participants of this study comprised 300 freshman students taking general English course chosen based on the availability sampling, three test developers, and seven university officials, administering the test. The main instrument is the teacher-made reading comprehension test to examine its validity, reliability, and differential item functioning (DIF). Furthermore, to examine the other modules of TFF, namely, access, administration, and social consequence, a questionnaire and semi-structured interviews with the test developers and test administrators were applied. In order to analyze the data, exploratory factor analysis is used to evaluate test validity employing Minitab software. Moreover, t test and ANOVA were used to examine the disparate impact of the test using SPSS package. Furthermore, the Mantel-Haenszel procedure is applied to determine the DIF, while coding the required formulas in R programming. The frequencies of different aspects of access and test administration are explained and consolidated by qualitative data gleaned from interview sessions. Examining the data with respect to TFF modules, it was concluded the test must be enhanced in terms of validity, while it is totally fair. The statistical procedure and the mixed research design implemented in the present study can be a sound model to be applied by test developers to enhance the test fairness of the exams.
Highlights
IntroductionAccording to Flaugher (1973), the theory of test fairness was shown to be a complex issue, a discovery that had immediate real-life implications, in a time of increased attention to the fair treatment of minority groups
Fairness is one of the key issues that concern people about any testing procedure
The purpose of the present study is to investigate the fairness of a locally designed English reading comprehension test which the students of higher education must pass as their general English basic course
Summary
According to Flaugher (1973), the theory of test fairness was shown to be a complex issue, a discovery that had immediate real-life implications, in a time of increased attention to the fair treatment of minority groups. In this way, different scholars such as Shohamy (2001), Weir (2005), and Fulcher and Davidson (2007) claimed that testing must be under careful examination because tests are mostly used for making high-. As Baharloo (2013) mentioned, fair judgment is an essential requirement for measurement professionals to be aware of the concept and its characteristics
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
