Comparison of the reliability and validity of scores from two concept-mapping techniques

Abstract

This paper reports the results of a study that compared two concept-mapping techniques, one high-directed, ''fill-in-the-map,'' and one low-directed, ''construct-a-map-from-scratch.'' We examined whether: (1) skeleton map scores were sensitive to the sample of nodes or linking lines to be filled in; (2) the two types of skeleton maps were equivalent; and (3) the two mapping techniques provided similar infor- mation about students' connected understanding. Results indicated that fill-in-the-map scores were not sensitive to the sample of concepts or linking lines to be filled in. Nevertheless, the fill-in-the-nodes and fill- in-the-lines techniques were not equivalent forms of fill-in-the-map. Finally, high-directed and low-directed maps led to different interpretations about students' knowledge structure. Whereas scores obtained under the high-directed technique indicated that students' performance was close to the maximum possible, the scores obtained with the low-directed technique revealed that students' knowledge was incomplete com- pared to a criterion map. We concluded that the construct-a-map technique better reflected differences among students' knowledge structure. fl 2001 John Wiley & Sons, Inc. J Res Sci Teach 38: 260 - 278, 2001 Concept maps have been used to assess students' knowledge structure, especially in science education (Novak, 1990). The justification for assessing students' knowledge structures is based on the idea that relating concepts that belong to the same domain is an important characteristic of scientific literacy (e.g., Bybee, 1996; Moore, 1995). Theory and research have shown that understanding a subject domain such as science is associated with a rich set of relations among important concepts in the domain (Novak, 1998; Novak & Gowin, 1984; Novak, Gowin, & Johansen, 1983; Novak & Ridley, 1988). We know, for example, that successful learners develop elaborate and highly integrated frameworks of related concepts (Mintzes, Wandersee, & Novak, 1997), just as experts do (Chi, Glaser, & Farr, 1988; Glaser, 1991). Research has shown that highly organized structures facilitate problem solving and other cognitive activities (e.g., generating explanations or rapidly recognizing meaningful patterns; Baxter, Elder, & Glaser,