Human Performance Validation of Simulators: Theory and Experimental Verification

Abstract

A theory of simulator validation employing a human performance correspondence approach is developed. The theory is based on the viewpoint that simulator validation is a problem of obtaining parallel measures in full-scale and in simulation and bringing these two sets of measures into correspondence. The approach taken is to adjust the simulator experimental conditions to obtain matching measure values between full-scale and simulation. Standard experimental design techniques can be used, but must be supplemented by additional discrimination techniques such as methods of correlation in repeated measure designs or confidence limit error measures in completely randomized designs. It is shown that the concept of performance validation is both α-level and sample size dependent, indicating that careful preliminary consideration should be given to the size of experiment to be performed. The theory was experimentally tested through the use of an instrumented full–scale test vehicle and a driving simulator. Data were collected on thirteen subjects in the full-scale vehicle and on seventy–four subjects in the simulator. Eight adjustment conditions in the simulator were used; they were obtained by varying two independent variables, random disturbance level and lateral display gain. Five driver performance measures for each condition in the simulator were compared to the same measures from the full–scale results. The simulator conditions which most closely matched the full–scale system were then determined. The results showed that for each performanc measure at least one simulator condition produced corresponding valid results. Overall, the results indicated that it is definitely possible to obtain performance validation through proper simulator adjustment.