The Development of a Theory-Based Instrument to Evaluate the Effectiveness of Continuing Medical Education

Abstract

To determine the psychometric properties of a theoretically based continuing medical education (CME) evaluation instrument examining attitudinal determinants of physicians' changes in medical practices after a CME intervention. The instrument's scales represented constructs from the theory of planned behavior. The authors based the template instrument on educational objectives of the CME intervention and adapted it to the clinical domain of preoperative breast cancer therapy. Development of the initial survey involved cognitive testing, pilot tests, and expert reviews. The authors asked 269 clinicians to complete the 35-item instrument before the CME intervention. Factor analysis and item analysis guided the development of the final six subscales: positive behavioral beliefs, negative behavioral beliefs, attitude toward the behavior, perceived behavior control (self-efficacy), subjective norms, and behavioral intention. Cognitive testing and pilot tests ensured the accuracy and clarity of the language and a reasonable survey length. Of the 269 clinicians, 168 (134 physicians) responded. Scales clustered according to the theoretical constructs. Items not loading in any subscales were eliminated. The final 25 items loaded on six subscales with loadings >0.54. Reliability for the subscales ranged from 0.73 to 0.93 (good for the scale development stage). The authors revised the instrument template and protocol after this initial study to increase the possibility of use in future CME evaluations. The levels of content and construct validity and reliability of the CME evaluation instrument are acceptable for evaluation of CME activities targeting physicians. Instruments adapted from this template could potentially evaluate future CME activities.