Deriving and measuring group knowledge structure from essays: The effects of anaphoric reference

Abstract

Essays are an important measure of complex learning, but pronouns can con- found an author's intended meaning for both readers and text analysis software. This descriptive investigation considers the effect of pronouns on a computer-based text anal- ysis approach, ALA-Reader, which uses students' essays as the data source for deriving individual and group knowledge representations. Participants in an undergraduate business course (n = 45) completed an essay as part of the course final examination. The investi- gators edited the essays to replace the most common pronouns (their, it, and they) with the appropriate referent. The original unedited and the edited essays were processed with ALA- Reader using two different approaches, sentence and linear aggregate. These data were then analyzed using a Pathfinder network approach. The average group network similarity values comparing the original to the edited essays were large (i.e., about 90% overlap) but the linear aggregate approach obtained larger values than the sentence aggregate approach. The linear aggregate approach also provided a better measure of individual essay scores (e.g., r= 0.74 with composite rater scores). This data provides some support that the ALA-