The priorities established among data sources when instructional designers revise written materials

Abstract

The objective of the study was to investigate the priorities instructional designers establish among data sources when they revise written materials and the relationship of their practice to standard models of formative evaluation. Two modules of printed instructional material were revised by each of eight experienced instructional designers under three treatment conditions, using a counterbalanced design. Data were collected using a think-aloud procedure. The think-aloud protocol was segmented, coded and analysed. Results show that instructional designers were significantly more likely to incorporate their own knowledge into revisions than to use feedback data, and when they did use feedback data, they preferred learner comments. They did not accurately assess their use of their own knowledge or of feedback data; they thought they used feedback data more than the results showed. These instructional designers reflected some aspects of the standard instructional design model when they revised instructional materials, but their experience has caused many of the procedures to become internalized, so they are invisible in the protocols. This research was funded in part by a grant from the Social Sciences and Humanities Research Council of Canada.