Validating a critical thinking ability questionnaire for EFL learners

Abstract

The present study validated a self-developed Critical Thinking Ability Questionnaire by analyzing data from 1169 valid questionnaires completed by students, as well as analyzing interview data from with four students and two experts. Through exploratory factor analysis, a seven-factor model was identified, including analysis, inference, argument evaluation, language construction, self-reflection and correction, cultural comparison, and credibility evaluation of language sources. Confirmatory factor analysis further supported the construct of the questionnaire, confirming that the construct of the questionnaire supports the adequacy and appropriateness of inferences made from the interpretation of questionnaire scores. The interviews provided additional evidence of the consistency between the critical thinking ability assessed by the questionnaire and the theoretical model. Experts interviewed also confirmed the relevance and coverage of the questionnaire, so the content of the questionnaire can support appropriate score interpretation. The evidence of criterion-related validity was supported by significant correlation coefficients. Additionally, the interpretation of questionnaire scores demonstrated satisfactory internal reliability and stability over time. With psychometrical and qualitative results, this research indicated that the CTA questionnaire is a sound instrument that demonstrated evidence of construct validity, content validity and criterion-related validity for Chinese high school students. Limitations and future directions are discussed.