Evaluating the Validity of Classroom Assessments in the Secondary EFL Curriculum

Abstract

This study investigated the validity of instructionally relevant classroom assessments embedded within a secondary English as a Foreign Language (EFL) curriculum. The study was conducted at a high school in Korea, including 148 12th graders learning EFL. For the validity analysis, two sets of achievement tests were chosen (i.e., multiple-choice listening tests for the midterm and final exams). The test items were qualitatively analyzed in relation to the national curriculum and instructional materials, and student test performance was also statistically analyzed. Results revealed that the two achievement tests were found to tap all curricular learning goals, albeit in an imbalanced manner due to the varied degree of coverage of the learning goals (cognitive validity). While the tests covered the target instructional contents, the degree of coverage and representativeness was again limited and inconsistent with the instructions (instructional validity). A statistical analysis of the test performance provided evidence of test validity but found a lack of consistency between the students’ listening ability and item difficulty levels (inferential validity). These findings suggest the types and forms of evidence required for test development and a validity argument within a learning-focused classroom assessment context.