Rank inversions in the scoring of examinations consisting of several subtests

Abstract

A test composed of k subtests is generally scored by assigning weights to the various subtests, defining the summary scores as the weighted sums of the subtest scores, and then ranking the summary scores of the group that was tested. Let a1,…,ak represent the weights for the respective subtests and ξ1,…,ξk the subtest scores for a particular individual; then the summary score for this individual is ∑i=1kaiξi. It has been the practice, particularly in psychological and educational tests, to replace the ith subtest score ξi by the standardized subtest score (ξi−μˆi)/si, where μˆi is the average subtest score and si is the calculated standard deviation for the group. The use of the standardized subtest scores in the definition of the summary score has the effect of changing the weight ai to ai/si,i=1,…,k. It is the purpose of this paper to show, under the assumption that the vectors of the subtest scores of those taking the test are i.i.d. normal random vectors in Rk, how the introduction of the standard deviations into the weighted sums affects the set of ranks based on the summary scores. For this purpose we compare the ranks based on the summary scores with weights (ai/si) to those based on the summary scores with weights (ai/σi), where σi is the true (unknown) standard deviation of the scores on the ith subtest, i=1,…,k. This represents an extension to a k-part examination of the author’s earlier result in the particular case k=2.