Training Test-Wiseness and Flawed Item Types

Abstract

This study investigated several current coaching practices used in training test-wiseness for analogy items in standardized test batteries. A three-group design was used which included a general test-taking, "encouragement" condition in addition to a no-training control group condition. The specific techniques used in training are described. Scholastic Aptitude Test (SAT) scores were obtained from university admission files to verify that no overall aptitude differences existed in the three conditions. Differences were observed for the coached group relative to the two control groups in terms of overall number of correct responses for the coached item types (analogies). No differences were found for the non-coached item types. Item difficulties for the three groups are also reported which show that several items were indeed made easier for individuals in the coached group. A qualitative analysis of the items made easier by coaching in terms of the training techniques used is given along with an analysis of the items that did not respond to coaching. Finally, a discussion of potentially flawed item types and item characteristics and suggestions for dealing with such flaws are given.