Human-machine Interface Design Research Articles

Characteristics of Rear-End Collisions: A Comparison between Automated Driving System-Involved Crashes and Advanced Driving Assistance System-Involved Crashes

With the increasing number of vehicles equipped with automated driving systems (ADS) being tested on public roads and the expanding market share of vehicles equipped with advanced driving assistance systems (ADAS), the number of ADS- or ADAS-involved crashes increases. Thus, it is necessary to investigate the distribution of ADS- and ADAS-involved crashes and the factors leading to them. The rear-end collision has been found to dominate ADS-involved crashes. However, no research has explored the conditions when ADS-involved rear-end collisions are more likely to happen and no research has investigated ADAS-involved rear-end crashes. Based on 130 ADS-involved crashes and 84 ADAS-involved crashes extracted from a dataset collected by the National Highway Traffic Safety Administration (NHTSA) between July 2021 and May 2022, this study explored the crash patterns, especially rear-end crashes, of ADS- and ADAS-controlled vehicles. Results show that rear-end collisions dominate both ADS- and ADAS-involved crashes, especially ADAS-involved crashes. The type of both ADS-involved and ADAS-involved crashes was affected by the speed of the ego-vehicle relative to the posted speed limit. Further, the type of ADS-involved crash was affected by the pre-crash movement of the crash partner, while the type of ADAS-involved crash was further associated with the road type. The findings can provide insights into the design of ADAS and ADS control algorithms, the external human-machine interface design of the vehicles with ADS or ADAS, and the training program of human road users to improve traffic safety in mixed traffic.

Automatic Water Level and Pressure Control System Prototype Design Using Programmable Logic Controller and Human Machine Interface

Water level control and pressure control are implemented in many at several industrial processing such as water dam, petrochemical processing plant, pharmacy and food processing plant, overhead tank, waste processing plant, electricity plant, etc. This study analyses a simulation and a needed components for automatically water level control system implementation using programmable logic controller (PLC). The water level control using PLC is designed to control the water level and prevent waste the water from a water tank. PLC is one of devices of automated controller which mostly used. Use of PLCs in industries are indispensable to control parameters which required high precision. In this research studies water level control system using PLC as primary controller and human machine interface (HMI) as remote controller that could visualize state of a system work (plant) in real time on computer screen and could gather field data as well. Main device for this research is Omron PLC CP2E type and touchscreen HMI Omron. At the prototype is used a water tank and a water container (to be one with the water resource), and in the HMI is showed the pages of home, control panel, and plant. From the result of the diagram ladder and the HMI design are got the system run well. The application from this research could be learning media in the system control laboratory for the subject of industrial control system and the others, related to it.

An integrating spherical fuzzy AHP and axiomatic design approach and its application in human–machine interface design evaluation

Human–machine interface (HMI) design evaluation is critical in interactive product development because it directly affects the cost of subsequent design and user experiments. The evaluation information of HMI design mainly depends on the subjective perception and preference of experts, especially the hesitancy degree is rarely considered. We propose an integrated spherical fuzzy AHP (SF-AHP) and spherical fuzzy axiomatic design (SF-AD) method to choose a reasonable HMI alternative, considering the potential risks caused by the hesitancy degree of experts. Firstly, we introduce a process of automatically repairing inconsistent spherical fuzzy preference relations (SFPR) for SF-AHP. After that, we build suitable evaluation criteria and calculate the criteria weight by the SF-AHP method. We further extend the axiomatic design to the spherical fuzzy environment and propose a multi-criteria decision-making (MCDM) method based on SF-AD to evaluate HMI alternatives. The case analysis results demonstrate the effectiveness of the proposed method in the HMI design evaluation process, and the sensitivity and comparative analysis results indicate that the proposed method is stable and reliable.

Expectancy or Salience?-Replicating Senders' Dial-Monitoring Experiments With a Gaze-Contingent Window.

In the 1950s and 1960s, John Senders carried out a number of influential experiments on the monitoring of multidegree-of-freedom systems. In these experiments, participants were tasked with detecting events (threshold crossings) for multiple dials, each presenting a signal with different bandwidth. Senders' analyses showed a nearly linear relationship between signal bandwidth and the amount of attention paid to the dial, and he argued that humans sample according to bandwidth, in line with the Nyquist-Shannon sampling theorem. The current study tested whether humans indeed sample the dials based on bandwidth alone or whether they also use salient peripheral cues. A dial-monitoring task was performed by 33 participants. In half of the trials, a gaze-contingent window was used that blocked peripheral vision. The results showed that, without peripheral vision, humans do not effectively distribute their attention across the dials. The findings also suggest that, when given full view, humans can detect the speed of the dial using their peripheral vision. It is concluded that salience and bandwidth are both drivers of distributed visual attention in a dial-monitoring task. The present findings indicate that salience plays a major role in guiding human attention. A subsequent recommendation for future human-machine interface design is that task-critical elements should be made salient.

Is text-based user manual enough? A driving simulator study of three training paradigms for conditionally automated driving

Training drivers about the automated driving system is one way for establishing safe and efficient driver interactions with a conditionally automated vehicle. This study investigated the effectiveness of three training paradigms—written, video (contains use cases), and interactive guides (support practices of use cases)—in a driving simulator study. Thirty participants with little or no prior experiences in driving automation were recruited and randomly assigned to participate in one of the training paradigms followed by a written test and two driving trips in a simulator. Explicit knowledge acquisition following training, driving performance during the simulated trips, and changes in acceptance of automation after training were analyzed. Results showed that: (1) there were no significant differences among the training groups in the declarative knowledge section of the test; however, the video guide group outperformed the written guide group in the procedural knowledge and strategic knowledge sections, and the interactive guide group outperformed the written guide group in the strategic knowledge; (2) Training type produced a significant effect on manual response times, but not visual response times, to non-scheduled takeover requests: the interactive guide group produced the shortest manual response times; (3) the interactive guide group performed better in takeover control during non-scheduled takeover scenario and (4) spent more time viewing the road during periods of low-reliability automated driving; (5) the written guide group reported decreased perceived safety after training. These findings suggest that the inclusion of use cases (in video and interactive guides) and providing hands-on practices (in interactive guide) have unique potentials in developing drivers’ knowledge in conditionally automated driving. This work has important implications for the effective design of conditionally automated driving training, as well as for the design of human-machine interface in automated vehicles to support users in developing and maintaining situation awareness.

Effects of collision warning characteristics on driving behaviors and safety in connected vehicle environments

With the emerging connected vehicle (CV) technologies, a novel in-vehicle omni-direction collision warning system (OCWS) is developed. For example, vehicles approaching from different directions can be detected, and advanced collision warnings caused by vehicles approaching from different directions can be provided. Effectiveness of OCWS in reducing crash and injury related to forward, rear-end and lateral collision is recognized. However, it is rare that the effects of collision warning characteristics including collision types and warning types on micro-level driver behaviors and safety performance is assessed. In this study, variations in drivers’ responses among different collision types and between visual only and visual plus auditory warnings are examined. In addition, moderating effects by driver characteristics including drivers’ demographics, years of driving experience, and annual driving distance are also considered. An in-vehicle human–machine interface (HMI) that can provide both visual and auditory warnings for forward, rear-end, and lateral collisions is installed on an instrumented vehicle. 51 drivers participate in the field tests. Performance indicators including relative speed change, time taken to accelerate/decelerate, and maximum lateral displacement are adopted to reflect drivers’ responses to collision warnings. Then, generalized estimation equation (GEE) approach is applied to examine the effects of drivers’ characteristics, collision type, warning type and their interaction on the driving performance. Results indicate that age, year of driving experience, collision type, and warning type can affect the driving performance. Findings should be indicative to the optimal design of in-vehicle HMI and thresholds for the activation of collision warnings that can increase the drivers’ awareness to collision warnings from different directions. Also, implementation of HMI can be customized with respect to individual driver characteristics.

Speculative Exploration on Future Sustainable Human-Machine Interface Design in Automated Shuttle Buses

Automated Shuttle buses (ASB) are considered an essential and sustainable direction for the future application of autonomous driving technology in public transportation. As the driver’s role gradually decreases and disappears, the Human–Machine Interface (HMI) for information exchange and communication between users and ASB takes a more prominent role and progressively becomes a hotspot in research. However, the unpredictability and complexity of autonomous driving, an exceptionally fast-growing technology, have hindered its future study. This work first reviewed related literature in three categories: internal, external, and station of the ASB. Secondly, the importance of systemic and speculative design is affirmed by exploring existing HMI designs for ASB. Thirdly, the concepts for ASB resulting from three parallel workshops were analyzed. Finally, online questionnaires and interviews completed the critical reflection and discussion. The results show that the introduction of tools and methods related to systemic and speculative design into the design process of the HMI for ASB may help designers to think critically about the future uncertainty of ASB and to deal with the complexity of the system.

Designing Gaze-Based Interactions for Teleoperation: Eye Stick and Eye Click

Perspective-taking and attentional switching are some of the ergonomic challenges that existing teleoperation human-machine interface designs need to address. This study developed two gaze interaction methods, the Eye Stick and the Eye Click, which were based on the joystick metaphor and the navigation metaphor, respectively, to be used in exocentric perspective teleoperation scenarios. We conducted two user studies to test the task performance and the subjective experience of the gaze interaction methods in a virtual ground vehicle teleoperation task. The results showed that compared with a traditional joystick design, the Eye Stick led to a shorter driving distance and the Eye Click led to less task time, and the gaze interaction methods had performance advantages in more difficult mazes. After multiple task sessions, the participants reported that the gaze interaction methods and the traditional joystick were similar in terms of task workload, perceived learnability, and satisfaction; however, the perceived usability of the Eye Stick was not as good as the Eye Click and the traditional joystick. In conclusion, both the Eye Stick and the Eye Click are feasible and promising gaze interaction methods for teleoperation applications with task performance advantages; however, more research is needed to optimize their user experience design.

A framework for usability evaluation of automated driving human-machine interface in context of UAE

As a future mobility enabler, vehicle teleoperation offers a viable option for efficiently leveraging the benefits of driverless cars, whilst fully automated cars (SAE 5) are not completely viable. Despite the huge amount of research on self-driving technology, the notion of riding in a fully automated vehicle is still sceptical. Hence necessitating the presence of a remote operator who tele monitors the vehicle and acquire control in case of failure beyond the capabilities of vehicle automation, such as misuse, disuse or mode confusion and ensuring safety and reliability. Keeping the current inclination towards automated vehicles and the launch of the United Arab Emirates” (UAE) first driverless, autonomous 12-seater shuttle, the researcher has suggested a low-fidelity basic prototype. To connect the vehicle’s automation and human teleoperation, the researcher has proposed a unique and interactive framework of Human-Machine Interface (HMI) to cater teleoperation in the context of automated vehicles (AVs) in the UAE. The design is optimised for the teleoperation of a highly automated shuttle by a public transportation control centre based on a logical and careful inquiry. After deriving a detailed analysis of requirements, a prototype is then created and refined until an interactive user-interface emerges. Fifteen control station experts are presented with a prototype for evaluation in a regular manner and solicited to work out three scenarios with disruptions in the system. The prototype is then evaluated using questionnaires and structured interviews. The results validate the suggested HMI design for the remote operation of a highly automated vehicle. Participants’ feedback shows a particularly high level of satisfaction in terms of perceived capacity, feasibility, usability approval and situational responsiveness. The findings also provide valuable information for an advanced interactive HMI design and pave the way for potential future directions in further research as well.

Improved Autonomous Trucker-Vehicle Dialogue under Critical Scenarios through fluid-HMI

Automation of vehicles not only provides greater safety, but also many previously unimagined opportunities, such as less inequality, less stress, and more meaningful activities while driving. However, the uptake and implementation of automated driving have been falling short of its promise, due to challenges in identifying safe and acceptable ways for humans to interact with automated vehicles. The human-machine interface (HMI) in vehicles plays a more critical role today than ever before. The main research question addressed in this study, as part of the EU-funded HADRIAN project, is: What steps need to be taken to achieve effective fluid HMI (f-HMI) design for improved driver-vehicle dialogue, especially in critical scenarios?

Ergonomic Reliability Assessment of VDT System for Operation Design Based on Improved BPNN and HCR under Special Circumstances

Ergonomic reliability plays a significant role in the safe operation of devices. With the spread of infectious diseases around the world, in work environments with high loads and high infection rates, medical staff work in a state of high self-protection. The use of visual display terminal (VDT) for medical equipment has undergone fundamental changes, and the traditional medical equipment human-machine interface design needs to be improved. After the completion of design and development, a VDT design enters the experimental testing stage, which has significant limitations for simulating the work of medical staff in the high-load and high-infection environments. The testing cost is high, and subjects face harsh conditions; thus, an ergonomic reliability model that can predict the use of VDT in such special high-infection and high-load circumstances must be established. An ergonomic reliability model based on an improved backpropagation neural network (BPNN) and human cognition reliability (HCR) is proposed for predicting and evaluating operation flows according to medical equipment VDTs. Firstly, a small data sample can be used to train BPNN to generate a network that can ensure suitable accuracy. To prevent the model from falling into local optimal solutions, the bat algorithm is introduced to improve the BPNN. Compared to a traditional BPNN, the superiority of the improved BPNN is clearly demonstrated. Secondly, the HCR method is used to analyze and highlight changes in the human factor reliability of VDTs for medical equipment in different time processes and operating processes according to BPNN prediction results, to provide a reference for selecting the optimal method. Finally, the validity and availability of the proposed method are verified through an eye tracker experiment and statistical analysis results.

A Computational Cognitive Model of Driver Response Time for Scheduled Freeway Exiting Takeovers in Conditionally Automated Vehicles.

This study develops a computational model to predict drivers' response time and understand the underlying cognitive mechanism for freeway exiting takeovers in conditionally automated vehicles (AVs). Previous research has modeled drivers' takeover response time in emergency scenarios that demand a quick response. However, existing models may not be applicable for scheduled, non-time-critical takeovers as drivers take longer to resume control when there is no time pressure. A model of driver response time in non-time-critical takeovers is lacking. A computational cognitive model of driver takeover response time is developed based on Queuing Network-Model Human Processor (QN-MHP) architecture. The model quantifies gaze redirection in response to takeover request (ToR), task prioritization, driver situation awareness, and driver trust to address the complexities of drivers' takeover strategies when sufficient time budget exists. Experimental data of a preliminary driving simulator study were used to validate the model. The model accounted for 97% of the experimental takeover response time for freeway exiting. The current model can successfully predict drivers' response time for scheduled, non-time-critical freeway exiting takeovers in conditionally AVs. This model can be applied to the human-machine interface design with respect to ToR lead time for enhancing safe freeway exiting takeovers in conditionally AVs. It also provides a foundation for future modeling work towards an integrated driver model of freeway exiting takeover performance.

Development and evaluation of a human machine interface to support mode awareness in different automated driving modes

With increasing implementation of automated driving technology it is expected that different automation modes will be present within the same vehicle and within a single trip. At all times during automated driving the driver needs to have ‘mode awareness’, which is an understanding of the automation mode and the corresponding responsibilities. Yet, research on HMI design to support mode awareness for multiple automation modes within a single vehicle and within a single trip is currently limited. The current work describes the development and evaluation of a Human Machine Interface (HMI) to support mode awareness while driving in different automation modes. The work exists of three phases: Phase 1 defines functional requirements for HMI design based on literature review and 5 experimental studies including 146 participants. Phase 2 implements the functional requirements in HMI design through expert and focus group sessions. Phase 3 evaluates and improves upon the HMI design employing virtual reality and the RITE (Rapid Iterative Testing and Evaluation) method with 18 participants. The result is a continuous and holistic HMI design creating mode awareness through ambience. Findings from Phase 3 and previous research indicate that this HMI is comprehended well, with a relatively low task load, and with a good experienced system usability. It is important to additionally evaluate the HMI design resulting from the current study in driving simulators and in on-road tests. Such tests will provide an opportunity to verify and expand on the current study’s findings and to contribute to guidelines for HMI design.

Optimization of Interface layout of electrical equipment using Heuristic Algorithm

Facility Layout Problem(FLP) is a major matter in the process of human-machine interface(HMI) design, which has a direct impact on how well operators perform their duties. Electrical equipment, which controls the power system, has a direct impact on how safely the system operates. The current optimization techniques prioritize improving the equipment’s performance characteristics at the expense of user aspects. This work aims to apply heuristic algorithms that introduce cognitive psychology and adopt the layout principle as a constraint condition to solve this problem. The improved analytic hierarchy process (AHP) is used to conduct the optimal design practice of the HMI of electrical switching equipment, develop the optimization mathematical model, and quantify the subjective and objective layout principles. Comparing the optimization results reveals that the genetic algorithm’s optimization impact is superior and the ant colony algorithm’s iterative pace is faster. The HMI’s optimized design based on a heuristic algorithm can improve the usability of interface operation, and has a certain reference for the optimization design of HMI of electrical equipment.

Transparency Assessment on Level 2 Automated Vehicle HMIs

The responsibility and role of human drivers during automated driving might change dynamically. In such cases, human-machine interface (HMI) transparency becomes crucial to facilitate driving safety, as the states of the automated vehicle have to be communicated correctly and efficiently. However, there is no standardized transparency assessment method to evaluate the understanding of human drivers toward the HMI. In this study, we defined functional transparency (FT) and, based on this definition, proposed a transparency assessment method as a preliminary step toward the objective measurement for HMI understanding. The proposed method was verified in an online survey where HMIs of different vehicle manufacturers were adopted and their transparencies assessed. Even though no significant result was found among HMI designs, FT was found to be significantly higher for participants more experienced with SAE Level 2 automated vehicles, suggesting that more experienced users understand the HMIs better. Further identification tests revealed that more icons in BMW’s and VW’s HMI designs were correctly used to evaluate the state of longitudinal and lateral control. This study provides a novel method for assessing transparency and minimizing confusion during automated driving, which could greatly assist the HMI design process in the future.

Managing Distraction Holistically through Driver Attention Support

As the characteristics of in-vehicle human-machine interfaces (HMIs), the driving task, and the expectations and behavior of drivers have evolved, so too should our thinking and approach to HMI design and evaluation. This panel will present background and perspectives on the current status, emerging needs, challenges, and opportunities in this area. A key focus for the panel is an emphasis on attention support. Detailed contextual description is provided below for reference to allow panelists to keep their opening remarks relatively brief to allow for substantive question and discussion time with the audience.

Evaluation of the optimal quantity of in-vehicle information icons using a fuzzy synthetic evaluation model in a driving simulator

In-Vehicle Information (IVI) features such as navigation assistance play an important role in the travel of drivers around the world. Frequent use of IVI, however, can easily increase the cognitive load of drivers. The interface design, especially the quantity of icons presented to the driver such as those for navigation, music, and phone calls, has not been fully researched. To determine the optimal number of icons, a systematic evaluation of the IVI Human Machine Interface (HMI) was examined using single-factor and multivariate analytical methods in a driving simulator. When one-way ANOVA was performed, the results showed that the 3-icon design scored best in subjective driver assessment, and the 4-icon design was best in the steering wheel angle. However, when a new method of analyzing the data that enabled a simultaneous accounting of changes observed in the dependent measures, 3 icons had the highest score (that is, revealed the overall best performance). This method is referred to as the fuzzy synthetic evaluation model (FSE). It represents the first use of it in an assessment of the HMI design of IVI. The findings also suggest that FSE will be applicable to various other HMI design problems.

Learning Spatio-Temporal Representations With a Dual-Stream 3-D Residual Network for Nondriving Activity Recognition

Accurate recognition of nondriving activity (NDA) is important for the design of intelligent human machine interface to achieve a smooth and safe control transition in the conditionally automated driving vehicle. However, some characteristics of such activities like limited-extent movement and similar background pose a challenge to the existing 3-D convolutional neural network based action recognition methods. In this article, we propose a dual-stream 3-D residual network, named DS3D residual network (ResNet), to enhance the learning of spatio-temporal representation and improve the activity recognition performance. Specifically, a parallel two-stream structure is introduced to focus on the learning of short-time spatial representation and small-region temporal representation. A two-feed driver behavior monitoring framework is further build to classify four types of NDAs and two types of driving behavior based on the driver's head and hand movement. A novel NDA dataset has been constructed for the evaluation, where the proposed DS3D ResNet achieves 83.35% average accuracy, at least 5% above three selected state-of-the-art methods. Furthermore, this study investigates the spatio-temporal features learned in the hidden layer through the saliency map, which explains the superiority of the proposed model on the selected NDAs.

The effect of information from dash-based human-machine interfaces on drivers' gaze patterns and lane-change manoeuvres after conditionally automated driving

The goal of this paper was to measure the effect of Human-Machine Interface (HMI) information and guidance on drivers' gaze and takeover behaviour during transitions of control from automation. The motivation for this study came from a gap in the literature, where previous research reports improved performance of drivers’ takeover based on HMI information, without considering its effect on drivers’ visual attention distribution, and how drivers also use the information available in the environment to guide their response. This driving simulator study investigated drivers’ lane-changing behaviour after resumption of control from automation. Different levels of information were provided on a dash-based HMI, prior to each lane change, to investigate how drivers distribute their attention between the surrounding environment and the HMI. The difficulty of the lane change was also manipulated by controlling the position of approaching vehicles in drivers’ offside lane. Results indicated that drivers' decision-making time was sensitive to the presence of nearby vehicles in the offside lane, but not directly influenced by the information on the HMI. In terms of gaze behaviour, the closer the position of vehicles in the offside lane, the longer drivers looked in that direction. Drivers looked more at the HMI, and less towards the road centre, when the HMI presented information about automation status, and included an advisory message indicating it was safe to change lane. Machine learning techniques showed a strong relationship between drivers' gaze to the information presented on the HMI, and decision-making time (DMT). These results contribute to our understanding of HMI design for automated vehicles, by demonstrating the attentional costs of an overly-informative HMI, and that drivers still rely on environmental information to perform a lane-change, even when the same information can be acquired by the HMI of the vehicle.

Design and evaluation of cooperative human–machine interface for changing lanes in conditional driving automation

This study analyzes the impact of cooperative human–machine interface designs on drivers’ trust in and interaction with automated driving systems (ADSs) during lane changes on highways. While drivers’ inappropriate trust in ADSs can affect their behavior toward the system, capability to detect inadequate system performance, and perception of surrounding traffic disturbances, their engagement in the automated process can improve their comprehension of the system and traffic conditions, which is necessary for the safe practice of automated driving. Forty drivers practiced conditional driving automation in a driving simulator and encountered traffic congestion on the main lane of a two-lane highway. Four ADS designs were proposed to bypass the congestion. ADS-1 detects the congestion and synchronizes the speed accordingly. ADS-2 requests the driver to resume manual control and overtake the congestion. ADS-3 requests the driver to push a button to let the system overtakes the congestion automatically. ADS-4 overtakes the congestion automatically after informing the driver, while the driver can cancel it by pushing a button within 6 s. In all these conditions, driver intervention was optional. Although the drivers preferred and trusted ADS-1 and ADS-2 more than ADS-3 and ADS-4, the results indicate significant improvements in the driving performance and system usage under ADS-3 and ADS-4. Driving with ADS-3 improved drivers’ engagement and reduced the requirement for control transfer compared with other systems. However, the time headway between the subject and adjacent vehicles indicated that lane changes were more critical under ADS-3 and ADS-4 than ADS-1 and ADS-2. Such deficiency of alignment between driver perception and safe behaviors has implications for the design of future studies and systems that need to balance satisfaction and safety. These observations are likely to improve driver interaction with automated vehicles.