Abstract
Objective: The human–machine interface (HMI) is a crucial part of every automated driving system (ADS). In the near future, it is likely that—depending on the operational design domain (ODD)—different levels of automation will be available within the same vehicle. The capabilities of a given automation level as well as the operator’s responsibilities must be communicated in an appropriate way. To date, however, there are no agreed-upon evaluation methods that can be used by human factors practitioners as well as researchers to test this.Methods: We developed an iterative test procedure that can be applied during the product development cycle of ADS. The test procedure is specifically designed to evaluate whether minimum requirements as proposed in NHTSA’s automated vehicle policy are met.Results: The proposed evaluation protocol includes (a) a method to identify relevant use cases for testing on the basis of all theoretically possible steady states and mode transitions of a given ADS; (b) an expert-based heuristic assessment to evaluate whether the HMI complies with applicable norms, standards, and best practices; and (c) an empirical evaluation of ADS HMIs using a standardized design for user studies and performance metrics.Conclusions: Each can be used as a stand-alone method or in combination to generate objective, reliable, and valid evaluations of HMIs, focusing on whether they meet minimum requirements. However, we also emphasize that other evaluation aspects such as controllability, misuse, and acceptance are not within the scope of the evaluation protocol.
Highlights
Automated driving (SAE level 2; Society of Automotive Engineers [SAE] 2018) in which the user is still required to control the vehicle in case of system limits and failures is already available by most manufacturers
In the context of automated driving, a variety of theoretical constructs related to the safe driver–vehicle interaction (DVI) such as trust, controllability, mode awareness, or usability could be used as criteria, because research has shown that they all pose challenges to the design and evaluation of human–machine interface (HMI) for automated driving system (ADS)
As an initial evaluation step, we propose a heuristic evaluation to be conducted by HMI experts (Naujoks et al 2019) to check whether the ADS HMI contains all necessary mode indicators and is in line with “voluntary guidance, best practices, and design principles published by SAE International, ISO, NHTSA ... ” (NHTSA 2017, p. 10)
Summary
Automated driving (SAE level 2; Society of Automotive Engineers [SAE] 2018) in which the user is still required to control the vehicle in case of system limits and failures is already available by most manufacturers. Higher levels of automation in which the driver is merely required to provide fallback performance of the driving task (conditional automation; SAE level 3) will be brought to the market in the few years This will likely create situations in which different levels of automation are present in the same vehicle, depending on the operational design domain (ODD; e.g., level 3 on the highway, level 2 on rural roads). In the context of driver distraction research, methods for investigating and ensuring the suitability of in-vehicle HMIs for use have been established For this purpose, standardized and generally accepted test and evaluation methods such as heuristic evaluations (e.g., Campbell et al 2016) and simulator study protocols (e.g., Alliance of Automobile Manufacturers 2006; NHTSA 2014) are available
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
