Impact of Frame-of-Reference and Behavioral Observation Training on Alternative Training Effectiveness Criteria in a Canadian Military Sample

Abstract

We examined the utility of frame-of-reference (FOR) training and behavioral observation training (BOT) for enhancing rating accuracy in a field study involving Canadian military personnel. Participants were randomly assigned to either (a) FOR training, (b) FOR training + BOT, or (c) control training. Participants were assessed on a variety of training program evaluation criteria, including rating accuracy, behavioral accuracy, and components of Kirkpatrick's (1959) typology of training effectiveness. Results indicated that FOR training led to significant enhancements in rating accuracy, and BOT-trained participants also produced significantly higher levels of behavioral accuracy compared to other conditions. However, contrary to predictions, the inclusion of BOT did not lead to significant increases in rating accuracy beyond FOR training alone. We also obtained evidence that trainee reactions to training predicted transfer of training and that trainees demonstrated knowledge of the frame-of-reference material imparted during training following a 4-month delay. We suggest that FOR and BOT may be useful approaches to rater training in field settings, although there may be some practical and logistical difficulties in developing and executing FOR training outside the laboratory.