Assessment of driving expertise using multiple choice questions including static vs. animated presentation of driving scenarios

Abstract

Novice drivers have a high collision risk compared to more experienced drivers. The optimization of driving assessment's validity should lead to a better identification of those examinees that need further training before solo driving. The aim of the present two studies was to find out whether the integration of animated traffic scenarios can enhance some quality aspects of selected multiple choice items of the official German driving test. For the first study, we developed a static version of a set of multiple choice questions each containing a computerized still picture of a traffic scenario and a dynamic version that included animated instead of still pictures of the same situations. Driving novices (n=57) and experts (n=63) were presented 22 items either in the static or in the dynamic version. Only the novices benefited from the dynamic presentation. Fifty novices and 50 experts participated in a second longitudinal study including three measurements with parallel tests of the dynamic testing materials at intervals of three to four months. The novices, who were currently attending driving school, improved over time whereas the experts’ performance remained stable. The results indicate that animation has positive effects on some quality aspects of driving assessment, but the problem of low criterion validity has to be addressed by the means of further approaches.