External Validity of TOEFL Section of Doctoral Entrance Examination in Iran: A Mixed Design Study

Abstract

External (generalization) validity is one aspect of construct validity (Messick, 1996) which deals with the inferences made on the basis of the test takers’ scores on a test. External validity is of particular importance in high stake tests such as tests of English for academic/specific purposes which are used to evaluate the test takers proficiency in general English for academic purposes. This study was an attempt to investigate the generalization validity of TOEFL tests administered at Iranian universities to select Ph. D candidates. In doing so, a mixed design study was applied. The data for the quantitative part was collected through a self assessment instrument consisting of personal information and 40 items designed on five point Likert scale. 450 doctoral students from different universities in Iran took part in the study. The data of the study were analyzed through descriptive and inferential statistics including principal component analysis, univariate analysis of variance and regression analysis (p=.05) approaches. The qualitative data was analyzed through content analysis. The results of the study indicated that there is a significant difference between the participants’ mean scores on TOEFL test and their means on academic language skills. Moreover, TOEFL scores did not significantly predict the test takers’ scores on the use of academic language in target language use situations. Therefore, TOEFL test developers should take the issue of generalizability into consideration while planning TOEFL tests.