AN ALTERNATIVE DATA COLLECTION DESIGN FOR EQUATING WITH VERY SMALL SAMPLES

Abstract

ABSTRACTA single group (SG) equating design with nearly equivalent test forms (SiGNET) design was developed by Grant (2006) to equate small volume tests. The basis of this design is that examinees take two largely overlapping test forms within a single administration. The scored items for the operational form are divided into mini‐tests called testlets. An additional testlet is created but not scored for the first form. If the scored testlets are Testlets 1‐6 and the unscored testlet is Testlet 7, then the first form is composed of Testlets 1‐6, the second form is composed of Testlets 2‐7, and Testlets 2‐6 are common to both test forms. They are administered as a single administered form, and when a sufficient number of examinees have taken the administered form for an SG equating, the second form (Testlets 2‐7) is equated to the first form (Testlets 1‐6) using SG equating. As evident, there are at least two merits of the SiGNET design over the nonequivalent groups with anchor test (NEAT) design. First, it facilitates the use of an SG equating design, which has the least random equating error, and second, it allows for the accumulation of sufficient data to equate the second form. Since the examinees scores are based on only the first form (i.e., the operational form), the two forms can be administered until sufficient data are collected to equate the second form. This study compared equatings under the SiGNET and NEAT designs and found reduced bias and error for the SiGNET design in very small sample size situations (e.g., N = 10 or 15). Implications for practice using the SiGNET design are also discussed.