The selection of upper and lower groups for the validation of test items.

Abstract

It is not intended here to discuss the general problem of item validation, but only that aspect of it thai arises when an upper and lower group are selected to serve as standard groups in the differentiation of test items. It is argued that the more indubitably it is known that the upper group is superior to the lower group, the more definitely can it be concluded that an item is valid by finding that the upper group is more successful in passing it than the lower group. If, in two situations, one in which the upper and lower groups are differentiated with high certainty and the other with little certainty, the proportion of passes (i.e., right answers) in the upper groups are equal and equally superior to the proportion of passes in the lower groups, we should believe that the item represented in the first situation is more valid than that of the second situation. Having available an initial group which is normally distributed with reference to a desired criterion, we set the problem of selecting upper and lower portions of this group which will be most efficient in the study of items, and their selection or rejection. The items in question are capable of two grades only, right or wrong. We further limit the issue by not here considering the interrelationship of items, a matter of first importance when the final test to be constructed is to contain more than one item. It is granted that the problem as set is too constricted to be real, but it is, nevertheless, believed that its solution is commonly pertinent to the handling of real item selection problems. The writer has stated that twenty-seven per cent should be selected at each extreme to yield upper and lower groups which are most indubitably different with respect to the trait in question. This article does not alter that conclusion but does provide a more available and somewhat improved derivation. Let us be given graduated scores on a test or trait from a sample of size N. For simplicity we shall consider N to be even, so that we may