The impact of response format on validating grammaticality judgment tests

Abstract

The present research aims to examine the influence of response format on the reliability and validity of grammaticality judgment tests. To this end, the grammaticality judgment test developed by Gass (1994) was selected as the instrument which was manipulated in terms of the response format while the original items remained intact. Thus, in the first phase of the study, a multiple-choice GJ test was constructed to be compared with the traditional form, i.e., the dichotomous type. The two tests were administered to a group of 110 students. With regard to validity which was arrived at by means of internal correlations proposed by Alderson, Clapham and Wall (1995). In the second phase, GJ tests were developed in ordinal and likert scales to which 49 students responded. The analysis unveiled the effect of response format on reliability. Concerning validity, which was examined via a post-test questionnaire, response format was selected as the second reason behind inconsistency in responses from participants’ view.