An investigation in evaluation issues for a simulation training programme

Abstract

AbstractRolfe (1991) and Parasuraman (1981) are critical of previous efforts by researchers to evaluate simulations given that very few authors in the field of simulation are actually considering validity, reliability or utility of simulations as training or educational methods.The Rank Ordering Exercise is designed as an activity which highlights evaluation issues. It was explained to 26 participants divided into 13 pairs. Each pair were asked to order 26 cards into a preferred chronological sequence—from the early stages of planning to the final stages of the project. Participants were given a scenario outlining a company which is experiencing complaints about the quality and standard of its customer care. A brief set of instructions asked participants to develop a training programme which incorporates a simulation, with the aim of building in evaluation components.The Rank Ordering Exercise raised awareness of internal and more importantly external validity issues. The authors noted an overall agreement about a preferred sequence of events and the exercise was effective in highlighting the need for the evaluation “checks” to be made during the planning stages of any research or consultancy project.