Human-machine Interface Design Research Articles

The Impact of Transparency on Driver Trust and Reliance in Highly Automated Driving: Presenting Appropriate Transparency in Automotive HMI

Automation transparency offers a promising way for users to understand the uncertainty of automated driving systems (ADS) and to calibrate their trust in them. However, not all levels of information may be necessary to achieve transparency. In this study, we conceptualized the transparency of the automotive human–machine interfaces (HMIs) in three levels, using driving scenarios comprised of two degrees of urgency to evaluate drivers’ trust and reliance on a highly automated driving system. The dependent measures included non-driving related task (NDRT) performance and visual attention, before and after viewing the interface, along with the drivers’ takeover performance, subjective trust, and workload. The results of the simulated experiment indicated that participants interacting with an SAT level 1 + 3 (system’s action and projection) and level 1 + 2 + 3 (system’s action, reasoning, and projection) HMI trusted and relied on the ADS more than did those using the baseline SAT level 1 (system’s action) HMI. The low-urgency scenario was associated with higher trust and reliance, and the drivers’ visual attention and NDRT performance improved after viewing the HMI, but not statistically significantly. The findings verified the positive role of the SAT model regarding human trust in the ADS, especially in regards to projection information in time-sensitive situations, and these results have implications for the design of automotive HMIs based on the SAT model to facilitate the human–ADS relationship.

디지털 적층 제조 장비의 사용자 중심 HMI 디자인 가이드 제안

With the advancement of metal additive manufacturing technology and the digitization of equipment, the need for user-centered design in HMI (Human-Machine Interface) has become increasingly apparent. However, research on design guides and usability test for industrial equipment has been lacking. Therefore, this study applied user-centered design principles to design and manufacture a prototype HMI for metal additive manufacturing equipment, followed by a usability test of the HMI prototype. Based on usability testing and in-depth interview analysis, this study proposes UX/UI design guidelines for user-centered industrial equipment HMI. The usability evaluation highlighted the need to improve the visibility of the system's real-time status, enhance interface intuitiveness for error prevention, and address emergency situations. It is crucial that the interface reflects both the visible and invisible states of the equipment, considering that the operation targets are mechanical equipment but the operating devices are HMI screens. Furthermore, for industrial equipment with numerous complex functions, it is essential to restrict or guide user actions through permission settings. Through the industrial PBF (Powder Bed Fusion) method metal additive manufacturing equipment HMI design guides proposed by this study, we anticipate the development of user-friendly and productivity-enhancing HMI designs.

Exploring factors in Human Cognition that affect Human Performance during Takeover in Autonomous Driving

Driver's mental load is an important influence on the design of the human-machine interface (HMI) for self-driving cars. During the autopilot-forced takeover process, aspects of the user's cognition, decision-making, attention, and emotional state affect the takeover performance. Therefore, the cognitive and psychological states of drivers must be considered in the HMI design of self-driving cars. And a good HMI design will reduce the user's mental burden and improve the takeover performance. In this study, a model affecting drivers' takeover performance was designed through human factors. And we refine the analysis from three aspects of cognitive load, emotional load, and environmental load by making subjective likert-style-questionnaires, derive the main influencing factors through the reliability analysis of the data, factor analysis, and other methods. This work finds that the clarity, comprehensiveness, and level of humanization of takeover messages, as well as the driver's arousal level when receiving the message, have the greatest impact on the driver's workload.

Designing for Trust: How Human-Machine Interface Can Shape the Future of Urban Air Mobility

This study investigates the influence of human-machine interface (HMI) design on passenger trust in autonomous electric vertical take-off and landing (eVTOL) vehicles. An immersive simulator-based experiment was conducted with 34 participants, exposing them to four HMI conditions: baseline, movement, hazard detection, and full information condition. As related measures of passenger trust, we collected self-reported measures including trust, perceived safety, perceived reliability, and intention to use. In addition, physiological measures including gaze behavior, electrodermal activity, and heart rate were collected. The results indicated that movement and hazard detection information improved passenger trust, suggesting that HMI design could play a crucial role in enhancing the acceptance of autonomous eVTOLs. In addition, gaze behavior showed a stronger relationship with self-reported trust than other physiological measures. The findings underscore the importance of HMI design in fostering passenger trust, which is critical to the success of urban air mobility.

Methodology based on machine learning through neck motion and POF-based pressure sensors for wheelchair operation

Polymer Optical Fiber (POF)-based sensors have gained recognition in recent years for biomedical applications because of their low cost, physical properties, and feasibility. A novel methodology is proposed here for classifying neck movements using POF- based pressure sensors and machine learning algorithms. To address this, signal pre-processing, feature extraction, and selection methods are implemented, considering variance, root mean square, and Hjorth parameters. Linear Discriminant Analysis, Support Vector Machine, k-Nearest Neighbors (kNN), and Decision Tree (DT) were used for classification. A maximum accuracy of 0.91 was obtained with kNN and DT for recognizing four neck movements by using the best discriminant nine features. These findings indicate that the proposed methodology is suitable for neck-motion classification using POF-based pressure sensors. Future work will focus on the implementation of this strategy for the design of intelligent Human Machine Interfaces based on electric-powered wheelchairs, which would allow for more independence for people with upper- and lower-limb disabilities.

Effects of Control Transition Strategies and Human-Machine Interface Designs on Driver Performance in Automated Driving Systems

Effects of Control Transition Strategies and Human-Machine Interface Designs on Driver Performance in Automated Driving Systems

Requirements for Human-Machine-Interaction Applications in Production and Logistics within Industry 5.0 – A Case Study Approach

This research paper examines the evolution of human-machine interfaces (HMIs) in the manufacturing and logistics sectors in the context of Industry 5.0 (I5.0). I5.0 introduces a human-centered paradigm, and as digitalization transforms processes in these domains, human-machine interaction is undergoing a notable shift. Yet, there is a lack of I5.0 HMI requirements and design guidelines. Through a literature review, the research identifies and investigates general requirements for HMIs in I5.0. Findings include ethical principles, ergonomics, adaptability, human-centered design, intuitive interaction, and effective collaboration. It also uses a case study approach to highlight potential strategies and discuss challenges. The results provide valuable insights into considerations for designing effective HMIs for I5.0 applications, enhancing the user experience, improving productivity, and facilitating successful human-machine interactions.

The Human–Machine Interface Design Based on sEMG and Motor Imagery EEG for Lower Limb Exoskeleton Assistance System

The Human–Machine Interface Design Based on sEMG and Motor Imagery EEG for Lower Limb Exoskeleton Assistance System

Automated HMI design as a custom feature in industrial SCADA systems

The adoption of emerging technologies under the Industry 4.0 paradigm has resulted in an increased quantity of data projected on the human-machine interface (HMI). This complexity is imposed in combination with users’ limited cognitive capabilities to interpret such data quantity. We foresee this limitation fails to address the human-centric approach of the Industry 5.0 paradigm. To overcome this challenge, an automated approach is proposed to enable user-guided ad hoc visualization forms, which ultimately increases users’ situational awareness of supervised industrial processes. In this paper, we present a framework for introducing automated HMI design as a custom feature of standard SCADA-based HMI functionality and showcase an example of how open technology may be used for such purposes, thus opening new avenues for rapid innovation on SCADA HMI. The framework enables integration of the user-initiated additions for loosely coupled visualization components configurable and integrated into the standard HMI runtime layer to meet changing user needs.

Characteristics of Rear-End Collisions: A Comparison between Automated Driving System-Involved Crashes and Advanced Driving Assistance System-Involved Crashes

With the increasing number of vehicles equipped with automated driving systems (ADS) being tested on public roads and the expanding market share of vehicles equipped with advanced driving assistance systems (ADAS), the number of ADS- or ADAS-involved crashes increases. Thus, it is necessary to investigate the distribution of ADS- and ADAS-involved crashes and the factors leading to them. The rear-end collision has been found to dominate ADS-involved crashes. However, no research has explored the conditions when ADS-involved rear-end collisions are more likely to happen and no research has investigated ADAS-involved rear-end crashes. Based on 130 ADS-involved crashes and 84 ADAS-involved crashes extracted from a dataset collected by the National Highway Traffic Safety Administration (NHTSA) between July 2021 and May 2022, this study explored the crash patterns, especially rear-end crashes, of ADS- and ADAS-controlled vehicles. Results show that rear-end collisions dominate both ADS- and ADAS-involved crashes, especially ADAS-involved crashes. The type of both ADS-involved and ADAS-involved crashes was affected by the speed of the ego-vehicle relative to the posted speed limit. Further, the type of ADS-involved crash was affected by the pre-crash movement of the crash partner, while the type of ADAS-involved crash was further associated with the road type. The findings can provide insights into the design of ADAS and ADS control algorithms, the external human-machine interface design of the vehicles with ADS or ADAS, and the training program of human road users to improve traffic safety in mixed traffic.

Automatic Water Level and Pressure Control System Prototype Design Using Programmable Logic Controller and Human Machine Interface

Water level control and pressure control are implemented in many at several industrial processing such as water dam, petrochemical processing plant, pharmacy and food processing plant, overhead tank, waste processing plant, electricity plant, etc. This study analyses a simulation and a needed components for automatically water level control system implementation using programmable logic controller (PLC). The water level control using PLC is designed to control the water level and prevent waste the water from a water tank. PLC is one of devices of automated controller which mostly used. Use of PLCs in industries are indispensable to control parameters which required high precision. In this research studies water level control system using PLC as primary controller and human machine interface (HMI) as remote controller that could visualize state of a system work (plant) in real time on computer screen and could gather field data as well. Main device for this research is Omron PLC CP2E type and touchscreen HMI Omron. At the prototype is used a water tank and a water container (to be one with the water resource), and in the HMI is showed the pages of home, control panel, and plant. From the result of the diagram ladder and the HMI design are got the system run well. The application from this research could be learning media in the system control laboratory for the subject of industrial control system and the others, related to it.

An integrating spherical fuzzy AHP and axiomatic design approach and its application in human–machine interface design evaluation

Human–machine interface (HMI) design evaluation is critical in interactive product development because it directly affects the cost of subsequent design and user experiments. The evaluation information of HMI design mainly depends on the subjective perception and preference of experts, especially the hesitancy degree is rarely considered. We propose an integrated spherical fuzzy AHP (SF-AHP) and spherical fuzzy axiomatic design (SF-AD) method to choose a reasonable HMI alternative, considering the potential risks caused by the hesitancy degree of experts. Firstly, we introduce a process of automatically repairing inconsistent spherical fuzzy preference relations (SFPR) for SF-AHP. After that, we build suitable evaluation criteria and calculate the criteria weight by the SF-AHP method. We further extend the axiomatic design to the spherical fuzzy environment and propose a multi-criteria decision-making (MCDM) method based on SF-AD to evaluate HMI alternatives. The case analysis results demonstrate the effectiveness of the proposed method in the HMI design evaluation process, and the sensitivity and comparative analysis results indicate that the proposed method is stable and reliable.

Expectancy or Salience?-Replicating Senders' Dial-Monitoring Experiments With a Gaze-Contingent Window.

In the 1950s and 1960s, John Senders carried out a number of influential experiments on the monitoring of multidegree-of-freedom systems. In these experiments, participants were tasked with detecting events (threshold crossings) for multiple dials, each presenting a signal with different bandwidth. Senders' analyses showed a nearly linear relationship between signal bandwidth and the amount of attention paid to the dial, and he argued that humans sample according to bandwidth, in line with the Nyquist-Shannon sampling theorem. The current study tested whether humans indeed sample the dials based on bandwidth alone or whether they also use salient peripheral cues. A dial-monitoring task was performed by 33 participants. In half of the trials, a gaze-contingent window was used that blocked peripheral vision. The results showed that, without peripheral vision, humans do not effectively distribute their attention across the dials. The findings also suggest that, when given full view, humans can detect the speed of the dial using their peripheral vision. It is concluded that salience and bandwidth are both drivers of distributed visual attention in a dial-monitoring task. The present findings indicate that salience plays a major role in guiding human attention. A subsequent recommendation for future human-machine interface design is that task-critical elements should be made salient.

Is text-based user manual enough? A driving simulator study of three training paradigms for conditionally automated driving

Training drivers about the automated driving system is one way for establishing safe and efficient driver interactions with a conditionally automated vehicle. This study investigated the effectiveness of three training paradigms—written, video (contains use cases), and interactive guides (support practices of use cases)—in a driving simulator study. Thirty participants with little or no prior experiences in driving automation were recruited and randomly assigned to participate in one of the training paradigms followed by a written test and two driving trips in a simulator. Explicit knowledge acquisition following training, driving performance during the simulated trips, and changes in acceptance of automation after training were analyzed. Results showed that: (1) there were no significant differences among the training groups in the declarative knowledge section of the test; however, the video guide group outperformed the written guide group in the procedural knowledge and strategic knowledge sections, and the interactive guide group outperformed the written guide group in the strategic knowledge; (2) Training type produced a significant effect on manual response times, but not visual response times, to non-scheduled takeover requests: the interactive guide group produced the shortest manual response times; (3) the interactive guide group performed better in takeover control during non-scheduled takeover scenario and (4) spent more time viewing the road during periods of low-reliability automated driving; (5) the written guide group reported decreased perceived safety after training. These findings suggest that the inclusion of use cases (in video and interactive guides) and providing hands-on practices (in interactive guide) have unique potentials in developing drivers’ knowledge in conditionally automated driving. This work has important implications for the effective design of conditionally automated driving training, as well as for the design of human-machine interface in automated vehicles to support users in developing and maintaining situation awareness.

Effects of collision warning characteristics on driving behaviors and safety in connected vehicle environments

With the emerging connected vehicle (CV) technologies, a novel in-vehicle omni-direction collision warning system (OCWS) is developed. For example, vehicles approaching from different directions can be detected, and advanced collision warnings caused by vehicles approaching from different directions can be provided. Effectiveness of OCWS in reducing crash and injury related to forward, rear-end and lateral collision is recognized. However, it is rare that the effects of collision warning characteristics including collision types and warning types on micro-level driver behaviors and safety performance is assessed. In this study, variations in drivers’ responses among different collision types and between visual only and visual plus auditory warnings are examined. In addition, moderating effects by driver characteristics including drivers’ demographics, years of driving experience, and annual driving distance are also considered. An in-vehicle human–machine interface (HMI) that can provide both visual and auditory warnings for forward, rear-end, and lateral collisions is installed on an instrumented vehicle. 51 drivers participate in the field tests. Performance indicators including relative speed change, time taken to accelerate/decelerate, and maximum lateral displacement are adopted to reflect drivers’ responses to collision warnings. Then, generalized estimation equation (GEE) approach is applied to examine the effects of drivers’ characteristics, collision type, warning type and their interaction on the driving performance. Results indicate that age, year of driving experience, collision type, and warning type can affect the driving performance. Findings should be indicative to the optimal design of in-vehicle HMI and thresholds for the activation of collision warnings that can increase the drivers’ awareness to collision warnings from different directions. Also, implementation of HMI can be customized with respect to individual driver characteristics.

Speculative Exploration on Future Sustainable Human-Machine Interface Design in Automated Shuttle Buses

Automated Shuttle buses (ASB) are considered an essential and sustainable direction for the future application of autonomous driving technology in public transportation. As the driver’s role gradually decreases and disappears, the Human–Machine Interface (HMI) for information exchange and communication between users and ASB takes a more prominent role and progressively becomes a hotspot in research. However, the unpredictability and complexity of autonomous driving, an exceptionally fast-growing technology, have hindered its future study. This work first reviewed related literature in three categories: internal, external, and station of the ASB. Secondly, the importance of systemic and speculative design is affirmed by exploring existing HMI designs for ASB. Thirdly, the concepts for ASB resulting from three parallel workshops were analyzed. Finally, online questionnaires and interviews completed the critical reflection and discussion. The results show that the introduction of tools and methods related to systemic and speculative design into the design process of the HMI for ASB may help designers to think critically about the future uncertainty of ASB and to deal with the complexity of the system.

Designing Gaze-Based Interactions for Teleoperation: Eye Stick and Eye Click

Perspective-taking and attentional switching are some of the ergonomic challenges that existing teleoperation human-machine interface designs need to address. This study developed two gaze interaction methods, the Eye Stick and the Eye Click, which were based on the joystick metaphor and the navigation metaphor, respectively, to be used in exocentric perspective teleoperation scenarios. We conducted two user studies to test the task performance and the subjective experience of the gaze interaction methods in a virtual ground vehicle teleoperation task. The results showed that compared with a traditional joystick design, the Eye Stick led to a shorter driving distance and the Eye Click led to less task time, and the gaze interaction methods had performance advantages in more difficult mazes. After multiple task sessions, the participants reported that the gaze interaction methods and the traditional joystick were similar in terms of task workload, perceived learnability, and satisfaction; however, the perceived usability of the Eye Stick was not as good as the Eye Click and the traditional joystick. In conclusion, both the Eye Stick and the Eye Click are feasible and promising gaze interaction methods for teleoperation applications with task performance advantages; however, more research is needed to optimize their user experience design.

A framework for usability evaluation of automated driving human-machine interface in context of UAE

As a future mobility enabler, vehicle teleoperation offers a viable option for efficiently leveraging the benefits of driverless cars, whilst fully automated cars (SAE 5) are not completely viable. Despite the huge amount of research on self-driving technology, the notion of riding in a fully automated vehicle is still sceptical. Hence necessitating the presence of a remote operator who tele monitors the vehicle and acquire control in case of failure beyond the capabilities of vehicle automation, such as misuse, disuse or mode confusion and ensuring safety and reliability. Keeping the current inclination towards automated vehicles and the launch of the United Arab Emirates” (UAE) first driverless, autonomous 12-seater shuttle, the researcher has suggested a low-fidelity basic prototype. To connect the vehicle’s automation and human teleoperation, the researcher has proposed a unique and interactive framework of Human-Machine Interface (HMI) to cater teleoperation in the context of automated vehicles (AVs) in the UAE. The design is optimised for the teleoperation of a highly automated shuttle by a public transportation control centre based on a logical and careful inquiry. After deriving a detailed analysis of requirements, a prototype is then created and refined until an interactive user-interface emerges. Fifteen control station experts are presented with a prototype for evaluation in a regular manner and solicited to work out three scenarios with disruptions in the system. The prototype is then evaluated using questionnaires and structured interviews. The results validate the suggested HMI design for the remote operation of a highly automated vehicle. Participants’ feedback shows a particularly high level of satisfaction in terms of perceived capacity, feasibility, usability approval and situational responsiveness. The findings also provide valuable information for an advanced interactive HMI design and pave the way for potential future directions in further research as well.

Improved Autonomous Trucker-Vehicle Dialogue under Critical Scenarios through fluid-HMI

Automation of vehicles not only provides greater safety, but also many previously unimagined opportunities, such as less inequality, less stress, and more meaningful activities while driving. However, the uptake and implementation of automated driving have been falling short of its promise, due to challenges in identifying safe and acceptable ways for humans to interact with automated vehicles. The human-machine interface (HMI) in vehicles plays a more critical role today than ever before. The main research question addressed in this study, as part of the EU-funded HADRIAN project, is: What steps need to be taken to achieve effective fluid HMI (f-HMI) design for improved driver-vehicle dialogue, especially in critical scenarios?

Ergonomic Reliability Assessment of VDT System for Operation Design Based on Improved BPNN and HCR under Special Circumstances

Ergonomic reliability plays a significant role in the safe operation of devices. With the spread of infectious diseases around the world, in work environments with high loads and high infection rates, medical staff work in a state of high self-protection. The use of visual display terminal (VDT) for medical equipment has undergone fundamental changes, and the traditional medical equipment human-machine interface design needs to be improved. After the completion of design and development, a VDT design enters the experimental testing stage, which has significant limitations for simulating the work of medical staff in the high-load and high-infection environments. The testing cost is high, and subjects face harsh conditions; thus, an ergonomic reliability model that can predict the use of VDT in such special high-infection and high-load circumstances must be established. An ergonomic reliability model based on an improved backpropagation neural network (BPNN) and human cognition reliability (HCR) is proposed for predicting and evaluating operation flows according to medical equipment VDTs. Firstly, a small data sample can be used to train BPNN to generate a network that can ensure suitable accuracy. To prevent the model from falling into local optimal solutions, the bat algorithm is introduced to improve the BPNN. Compared to a traditional BPNN, the superiority of the improved BPNN is clearly demonstrated. Secondly, the HCR method is used to analyze and highlight changes in the human factor reliability of VDTs for medical equipment in different time processes and operating processes according to BPNN prediction results, to provide a reference for selecting the optimal method. Finally, the validity and availability of the proposed method are verified through an eye tracker experiment and statistical analysis results.