Validity Of Driving Simulator Research Articles

Establishing driving simulator validity: drawbacks of null-hypothesis significance testing when compared to equivalence tests and Bayes factors

The validity of driving simulators is a topic that has been addressed ever since their first use. In practice, a comparison of field data and simulator data is often used for validation purposes. A systematic literature search was conducted which identified large heterogeneity in both study designs and statistical methods used in research on simulator validity. Most of the included studies concluded simulator validity (thus equivalence of field data and simulator data) from non-significant results in Null-Hypothesis Significance Testing (NHST), which is a well-known yet somewhat controversial practice. In this work, the consequences of both study design choice and the choice of the statistical method are outlined and alternatives such as using Bayes factor hypothesis testing and equivalence tests are discussed and applied to a real data example. The present paper is intended to shed light on the influence of the statistical method on conclusions drawn from experimental data. For research on driving simulator validity, we highly encourage the use of equivalence tests or Bayes factor hypothesis tests, in particular when evidence of the data suggests that an effect might be absent.

Drivers’ behaviour when overtaking cyclists on rural roads: Driving simulator validation using naturalistic data

Driving simulator is a useful tool for obtaining data on driver behaviour efficiently and quickly. However, to ensure the reliability of the data gathered by the simulator, it is necessary to check the differences between drivers’ behaviour in the simulator and in reality. In this study, an existing two-lane rural road was replicated on the driving simulator under the same traffic conditions of groups of cyclists and oncoming motorised vehicles. For this purpose, a naturalistic field data collection was developed on the real road using instrumented bicycles and static video recordings. A total of 30 volunteers participated in the driving simulator tests. The objective validation of the driving simulator was based on three operational variables: average travel speed, overtaking vehicle speed, and lateral clearance. As a result, higher average travel speeds and lower lateral clearances were obtained in the real world compared to those observed in the simulator. It was also found that overtaking vehicle speed depends on the group of cyclists. Overall, the data obtained in the field and in the driving simulator did not present statistically significant differences. The analysis of drivers’ perception in the simulator tests concluded that the simulator reflected reality in an accurate way, achieving the subjective validation of the driving simulator. Thus, this study validates the driving simulator for bicycle safety research on rural roads.

Driving Simulator Validity of Driving Behavior in Work Zones

Driving simulation is an efficient, safe, and data-collection-friendly method to examine driving behavior in a controlled environment. However, the validity of a driving simulator is inconsistent when the type of the driving simulator or the driving scenario is different. The purpose of this research is to verify driving simulator validity in driving behavior research in work zones. A field experiment and a corresponding simulation experiment were conducted to collect behavioral data. Indicators such as speed, car-following distance, and reaction delay time were chosen to examine the absolute and relative validity of the driving simulator. In particular, a survival analysis method was proposed in this research to examine the validity of reaction delay time. The result indicates the following: (1) most indicators are valid in driving behavior research in the work zone. For example, spot speed, car-following distance, headway, and reaction delay time show absolute validity. (2) Standard deviation of the car-following distance shows relative validity. Consistent with previous researches, some driving behaviors appear to be more aggressive in the simulation environment.

Driving Simulator Validation for In-Vehicle Human Machine Interface Assessment

Driving simulator validation is an important and ongoing process. Advances in in-vehicle human machine interfaces (HMI) mean there is a continuing need to reevaluate the validity of use cases of driving simulators relative to real world driving. Along with this, our tools for evaluating driver demand are evolving, and these approaches and measures must also be considered in evaluating the validity of a driving simulator for particular purposes. We compare driver glance behavior during HMI interactions with a production level multi-modal infotainment system on-road and in a driving simulator. In glance behavior analysis using traditional glance metrics, as well as a contemporary modified AttenD measure, we see evidence for strong relative validity and instances of absolute validity of the simulator compared to on-road driving.

Systematic review of driving simulator validation studies

Driving simulators are a common tool for researching driver behaviour, providing practical, safe, and controlled environments. Despite their frequent use in research, there is relatively little evidence confirming their validity (i.e., how accurately they represent or reproduce real-world driving). Moreover, there is inconsistency in both the types of simulators used, and the operationalisation of “real-world” driving in validations. This systematic review was undertaken to evaluate the evidence regarding driving simulator accuracy when compared with real-world driving. The review included 44 studies reporting a direct comparison between simulated driving and on-road driving in a vehicle. Measures reported for comparison varied but included mean speed, speed variability, lateral position, overall driving performance, and number of driving errors. Simulators in approximately half of the studies achieved absolute or relative validity, whereas one third produced non-valid results. To understand this further, the fidelity of simulators was considered, however this further clouded our understanding as the relationship between simulator fidelity and validity was not straightforward. The findings suggest that the reporting of driving simulator studies requires improvement, particularly around the validation evidence associated with the simulator, the specific details of the simulated driving environment, and the outputs of statistical analyses. Guidelines are proposed for future research to ensure consistency in the conduct, and reporting, of simulator-based research.

Assessing Driving Simulator Validity: A Comparison of Multi-Modal Smartphone Interactions across Simulated and Field Environments

The use of a driving simulator as a tool to evaluate secondary task performance elicits the question of simulator validity. After upgrading an existing driving simulator from a medium-fidelity to a high-fidelity configuration with a new software environment, a study was run to benchmark this simulator against previously published highway-driving data. A primary goal was to assess relative and absolute validity in a simulated highway environment. Data from 72 participants who performed manual and voice-based contact dialing tasks with a center-stack-mounted smartphone in either the driving simulator or driving on the highway in one of two vehicles is considered. This analysis compared secondary task demand between the simulator and on-road vehicles by primarily considering driver off-road glance behavior. Mean total eyes-off-road time, mean single-glance duration, and the number of long off-road glances showed similar patterns relative to the manual versus voice-based tasks in the simulator and the two on-road vehicles. A driving performance metric, percentage change of standard deviation of velocity, showed differing results between the simulator and on-road vehicles. It is concluded that these data make a strong argument for relative validity, and in some cases absolute validity, for this simulator for studying glance behavior associated with in-vehicle devices under a highway-driving scenario.

Driving simulator validation for speed research on curves of two-lane rural roads

The numerous advantages of driving simulators make them a useful research tool. However, validation studies need to be conducted before extrapolating driving simulator results from the laboratory to a real-world driving environment. In this study, validation of mean speeds on curves obtained in a driving simulator was performed by monitoring speed on corresponding real-world curves. A mountainous two-lane rural road in Iran was simulated and the driving speeds of 30 participants were collected at the beginning, middle and end of five curves in the simulator. The driving speeds of 40 different participants were also measured at specified points of five curves in the real road using a radar gun. The curves studied included a horizontal curve on a positive grade, three combined crest curves and one combined sag curve. The results of bilateral z-tests indicated that both experiments belonged to the same population and independent t-tests indicated equality of variances for all curves. In most curves, except the combined crest curve with a radius of 300 m, the mean speeds in the driving simulator were higher than the real-road condition. The trends in speed variation from the beginning to the end of the curve were similar in both experiments.

Driving simulator validation of driver behavior with limited safe vantage points for data collection in work zones

IntroductionThis study is aimed at validating a driving simulator (DS) for the study of driver behavior in work zones. A validation study requires field data collection. For studies conducted in highway work zones, the availability of safe vantage points for data collection at critical locations can be a significant challenge. A validation framework is therefore proposed in this paper, demonstrated using a fixed-based DS that addresses the issue by using a global positioning system (GPS). MethodsThe validation of the DS was conducted using objective and subjective evaluations. The objective validation was divided into qualitative and quantitative evaluations. The DS was validated by comparing the results of simulation with the field data, which were collected using a GPS along the highway and video recordings at specific locations in a work zone. The constructed work zone scenario in the DS was subjectively evaluated with 46 participants. ResultsThe objective evaluation established the absolute and relative validity of the DS. The mean speeds from the DS data showed excellent agreement with the field data. The subjective evaluation indicated realistic driving experience by the participants. Practical applicationsThe use of GPS showed that continuous data collected along the highway can overcome the challenges of unavailability of safe vantage points especially at critical locations. Further, a validated DS can be used for examining driver behavior in complex situations by replicating realistic scenarios.

Protocol for Transferability of Driver Simulator Results to Real Traffic Conditions

The current manuscript presents the research protocol that has been developed in order to enable the valid transferability of driver simulator results in real traffic conditions. It responds to the weaknesses that have been recognised in the driving simulator validity research field, with respect to the validity of the experimental process being followed as well as to the methodology applied for the comparative analysis of the collected measurements. A case study concerning a semi-dynamic driving simulator is presented. The research hypotheses, the experimental plan and the basic conditions of the analysis methodology that have supported it are described. Trials have been conducted in a semi-dynamic simulator and on-road with 36 drivers (12 trainees, 12 novice and 12 experienced), in “following vehicle” and “free driving” scenarios in highway, rural and urban roads in order to prove the established research hypotheses for a series of driving behaviour metrics.

The Study of Driving Simulator Validation for Physiological Signal Measures

The current research aimed at validating the driving simulator from Beijing University of Technology in order to verify its usefulness in simulated experiment study based on the physiological signals. Researchers often account for behavioral validation through comparing speed and lane position of field and simulator. In this study, we compared the subjects’ electroencephalogram and electrocardiogram of field and simulator. Results showed that driving simulation is absolutely effectiveness in straight sections and large radius corners sections, and the index is not need calibrated, the data can be used as the experimental results. Otherwise, it is necessary to calibrate the index.

Driving simulator validation with hazard perception

Abstract How should we assess the comparability of driving on a road and “driving” in a simulator? If similar patterns of behaviour are observed, with similar differences between individuals, then we can conclude that driving in the simulator will deliver representative results and the advantages of simulators (controlled environments, hazardous situations) can be appreciated. To evaluate a driving simulator here we compare hazard detection while driving on roads, while watching short film clips recorded from a vehicle moving through traffic, and while driving through a simulated city in a fully instrumented fixed-base simulator with a 90 degree forward view (plus mirrors) that is under the speed/direction control of the driver. In all three situations we find increased scanning by more experienced and especially professional drivers, and earlier eye fixations on hazardous objects for experienced drivers. This comparability encourages the use of simulators in drivers training and testing.

Driving Simulator Validation for Nighttime Construction Work Zone Devices

Research was conducted to ascertain the validity of a driving simulator in determining the effectiveness of temporary traffic control devices in a work zone at night. The research was conducted through a field study and a simulator study. The field study examined speeds at six sites. The simulator study involved 127 human subjects. Spot speeds were observed at three locations in a freeway work zone. Locations of the speed studies were at the beginning of the work zone near the transition area, in the middle of the work zone, and at the end of the work zone near the downstream taper. Statistical analyses were conducted to determine whether participants of the study performed differently in the simulator, compared with in the field. Research results established that because of the motorist's perceived risk of work zones, driving simulators may not replicate mean speeds observed in the field.

Technical report: Literature survey on driving simulator validation studies

The use of real-time driving simulators for research and training of vehicle operators has increased significantly in recent years. There is a wide range of complexity among simulators currently being used around the world. An important issue to consider when implementing any simulator is validity. This paper presents different interpretations of validity in the context of simulators and vehicle simulation models found in the literature. Examples of different approaches to validating simulators and vehicle simulation models from studies found in the literature are presented.

Driving simulator validation for speed research

The behavioral validation of an advanced driving simulator for its use in evaluating speeding countermeasures was performed for mean speed. Using mature drivers, 24 participants drove an instrumented car and 20 participants drove the simulator in two separate experiments. Participants drove on roads which contained transverse rumble strips at three sites, as well as three equivalent control sites. The three pairs of sites involved deceleration, and were the approaches to stop sign intersections, right curves, and left curves. Numerical correspondence (absolute validity), relative correspondence (or validity), and interactive (or dynamic) relative validity were analyzed, the latter using correlations developed from canonical correlation. Participants reacted to the rumble strips, in relation to their deceleration pattern on the control road, in very similar ways in both the instrumented car and simulator experiments, establishing the relative validities. However, participants generally drove faster in the instrumented car than the simulator, resulting in absolute validity not being established.

Driving Simulator Validity: Some Considerations

The validity of driving simulators for behavioral research is discussed. The concept of validity is introduced and explained, and a survey of validation studies follows, in the TNO driving simulator and others, comparing field and simulator study results. Results for mid-level driving simulators show that generally absolute validity of route choice behavior is obtained and relative validity of speed and lateral control behavior is obtained. There is evidence suggesting that for a number of applications the presence of a moving base and possibly a higher image resolution might increase the validity of a driving simulator.

Driving Simulator Validity: Some Considerations

The validity of driving simulators for behavioral research is discussed. The concept of validity is introduced and explained, and a survey of validation studies follows, in the TNO driving simulator and others, comparing field and simulator study results. Results for mid-level driving simulators show that generally absolute validity of route choice behavior is obtained and relative validity of speed and lateral control behavior is obtained. There is evidence suggesting that for a number of applications the presence of a moving base and possibly a higher image resolution might increase the validity of a driving simulator.