Investigating the Construct Validity of the FCE Reading Paper in Iranian EFL Context

Abstract

Validity studies on language proficiency tests have attracted many researchers in the last decades. Most of such studies try to investigate skills assessed by the items in the tests, hence the construct validity of the test. The present study took the same approach and therefore aimed to investigate the construct validity of the reading paper of the First Certificate in English (FCE) in the Iranian EFL context. The research addressed the following three questions: (1) Do the majority of the Iranian EFL expert judges agree on the skills measured by the items in the FCE reading paper? (2) Do the majority of the Iranian EFL undergraduates come to agreement on the skills measured by the items in the FCE reading paper? (3) Does exploratory factor analysis support that the FCE reading paper assesses the same reading skills claimed by its developing board, i.e. UCLES, in the context of Iranian EFL undergraduates? The present study deployed triangulated approach in collecting and analyzing the data. It attempted to use both qualitative (i.e. experts’ and test takers’ judgment) and quantitative (i.e. Factor Analysis) approaches to address the above questions. The findings revealed that there was not a substantial agreement among expert judges on the skills claimed to be assessed by the items in the FCE reading paper nor could any significant agreement be observed amongst the test takers on the skills being measured by such items in the FCE reading paper. Finally, exploratory factor analysis revealed similar findings as those in the judgmental phase of the study. The individual items in the FCE reading paper did not confirm the claims by the test developers.