Measuring Motivational Characteristics of Courses: Applying Kellerʼs Instructional Materials Motivation Survey to a Web-Based Course

Abstract

The Instructional Materials Motivation Survey (IMMS) purports to assess the motivational characteristics of instructional materials or courses using the Attention, Relevance, Confidence, and Satisfaction (ARCS) model of motivation. The IMMS has received little use or study in medical education. The authors sought to evaluate the validity of IMMS scores and compare scores between standard and adaptive Web-based learning modules. During the 2005-2006 academic year, 124 internal medicine residents at the Mayo School of Graduate Medical Education (Rochester, Minnesota) were asked to complete the IMMS for two Web-based learning modules. Participants were randomly assigned to use one module that adapted to their prior knowledge of the topic, and one module using a nonadaptive design. IMMS internal structure was evaluated using Cronbach alpha and interdimension score correlations. Relations to other variables were explored through correlation with global module satisfaction and regression with knowledge scores. Of the 124 eligible participants, 79 (64%) completed the IMMS at least once. Cronbach alpha was >or=0.75 for scores from all IMMS dimensions. Interdimension score correlations ranged 0.40 to 0.80, whereas correlations between IMMS scores and global satisfaction ratings ranged 0.40 to 0.63 (P<.001). Knowledge scores were associated with Attention and Relevance subscores (P=.033 and .01, respectively) but not with other IMMS dimensions (P>or=.07). IMMS scores were similar between module designs (on a five-point scale, differences ranged from 0.0 to 0.15, P>or=.33). These limited data generally support the validity of IMMS scores. Adaptive and standard Web-based instructional designs were similarly motivating. Cautious use and further study of the IMMS are warranted.