Developmental Evaluation of a Distributed Online Simulation

Abstract

The formative evaluation of developing training systems presents problems at many levels, and development must be based upon training effectiveness information. Acceptance of subjective estimates of training effectiveness requires that the providers reasonably represent the user population and have some relevant expertise. The U.S. Army is developing a distributed simulation using online gaming technology to address conventional and asymmetric warfare, and requires feedback for development. Formative evaluation by military personnel is being used to address interface usability, minimal acceptable levels of fidelity, and potential effectiveness in training. Results from an Army National Guard and an Active Army unit are presented, with data supporting their acceptability as representative evaluators. Their results indicate considerable potential for the system to meet new asymmetric training needs and optimize Soldiers' training time. The effort also demonstrates the difficulty of establishing validity of users in developmental evaluation.