Alchemy and beyond: Indexing the Defining Issues Test.

Abstract

For over 20 years, the Defining Issues Test (DIT) has used the P index. In view of criticisms, a search has been underway for a new index. The authors propose a working definition of construct validity, systematically reanalyze existing data sets (classic studies) with new indexes, and make comparisons to trends obtained using the P index. The criteria for construct validity are (a) sensitivity to educational interventions, (b) differentiation of age-education al groups, (c) upward movement in longitudinal studies, (d) correlations with moral comprehension, (e) correlations with prosocial behavior, and (f) correlations with civil libertarian attitudes. As meta-analysis demonstrates, a new index, N2, generally outperforms the P index. In the early 1970s, Larry Kohlberg found amusement by comparing the research project on the Defining Issues Test (DIT) to alchemy. The alchemist's dream of the middle ages had been to transmute the base metals into gold. At the time, Kohlberg was beginning work on revising his scoring system and was mindful of the complexities involved in analyzing moral judgments and the arduous work required of a scorer. Kohlberg pointed out similarities between alchemy and the attempt to derive a measure of moral judgment from a multiple-choic e test. Obtaining moral development scores by simply asking participants to rate or rank statements seemed too good to be true—it was like trying to turn lead into gold. Nevertheless he was supportive of the exploration of new sources of information on moral judgment (Kohlberg, 1979), although he did make sure that we were alert to the possible problems with such an approach. Years later, once we had developed computer programs to score the DIT, we joked with Kohlberg about doing morality research untouched by human hands'1—the very thought of which he also found preposterously amusing. For over 20 years, DIT researchers have relied on the P score to index moral judgment and for that long we have tried to find a better index. Not only did we aspire to find a