Examining the Impact of Covariates on Anchor Tests to Ascertain Quality Over Time in a College Admissions Test

Abstract

We propose a comprehensive procedure for the implementation of a quality control process of anchor tests for a college admissions test with multiple consecutive administrations. We propose to examine the anchor tests and their items in connection with covariates to investigate if there was any unusual behavior in the anchor test results over time and if the test results differed for different groups of test takers. Descriptive statistics, ANOVA, and linear mixed effect models were used to examine the impact of covariates on the anchor test over time. Descriptive statistics and logistic regression were used to examine the quality of each anchor item over time. It is concluded that examining the impact of the test takers' covariates at different administrations can help in the process of providing fair tests for all test takers. The results are discussed with recommendations how it can be used in other large testing programs.