Human-machine Interface Design Research Articles

Workplace Design of Selected Rice Transplanters for Farm-women Operators

Background: Modern self-propelled devices consider human factors such as space constraints, control placement, line of sight, comfort and ease of operation. These factors are linked to the operator’s anthropometric and biomechanical parameters. Effective human-machine interface design can enhance productivity, comfort and safety. Methods: An anthropometric survey was conducted among 450 female agricultural workers across three zones of Kerala. The dimensions and clearances of the operator workplace for a selected transplanter were measured based on the seat index and reference points. The data were compared with the anthropometric dimensions to assess the suitability of the transplanters for women operators. Result: Based on the anthropometric data and spatial requirements, seat designs were proposed for Yanji and Redlands transplanters. Recommendations for the operator’s seat include a 39 cm backrest height, 38 cm bottom width, 18 cm open area, 40 ± 3.4 cm length, 43 cm top width, 10 cm seat cushion height and 8 cm backrest cushion height. A 35 cm diameter circular seat is recommended for the helper.

Comparing User Acceptance in Human–Machine Interfaces Assessments of Shared Autonomous Vehicles: A Standardized Test Procedure

Human–Machine Interfaces (HMIs) in autonomous driving technology have recently gained significant research interest in public transportation. However, most of the studies are biased towards qualitative methods, while combining quantitative and qualitative approaches has yet to receive commensurate attention in measuring user acceptance of design outcome evaluation. To the best of our knowledge, no standardized test procedure that combines quantitative and qualitative methods has been formed to evaluate and compare the interrelationships between different designs of HMIs and their psychological effects on users. This paper proposes a practical and comprehensive protocol to guide assessments of user acceptance of HMI design solutions. We first defined user acceptance and analyzed the existing evaluation methods. Then, specific ergonomic factors and requirements that the designed output HMI should meet were identified. Based on this, we developed a protocol to evaluate a particular HMI solution from in- and out-of-vehicle perspectives. Our theoretical protocol combines objective and subjective measures to compare users’ behavior when interacting with Autonomous Vehicles (AVs) in a virtual experimental environment, especially in public transportation. Standardized testing procedures provide researchers and interaction designers with a practical framework and offer theoretical support for subsequent studies.

Research on Assessment Method for Cognitive Influence Factors of Nuclear Power Plant Operators Based on Fuzzy Analytic Hierarchy Process and Deep Learning of Electroencephalogram Signals

This study explores the factors influencing the cognitive processes of operators in digital nuclear power plants, with a focus on the correlation between these factors and electroencephalogram (EEG) features. Initially, based on expert consultations, seven factors were considered: stress, time, fatigue, procedural complexity, user interface experience, procedural clarity, and efficiency. From these, four were identified as the most crucial for each stage of the cognitive process, highlighting their significant roles in influencing cognitive performance and potentially correlating with distinct EEG characteristics. These were assessed using the fuzzy analytic hierarchy process (FAHP) to determine the weightings of influences across the cognitive stages of monitoring, decision making, and execution. Employing a simulated scenario of a steam generator tube rupture, subjective questionnaires were utilized to gauge participant perceptions of influencer impacts at each stage, calculating human factors fuzzy synthetic values. Concurrently, EEG signals were segmented by operational steps, extracting around 114 features across the time, frequency, and time-frequency domains, which were then dimensionally reduced to 17 principal components via adaptive principal components analysis (APCA). A correlation analysis was performed between the human factors fuzzy synthetic values and the APCA-reduced EEG features of participants. Subsequently, the EEG feature columns of the eight selected participants were used as inputs to construct a transformer-based self-attention network model to evaluate the participants’ human factors fuzzy comprehensive values. The findings confirm the transformer model’s efficacy in assessing these values, evidencing a significant correlation between the EEG features and human factors fuzzy synthetic values. Integrating FAHP with machine learning methodologies, this model proficiently estimated operators’ cognitive states during various cognitive processes, significantly enhancing human-machine interface design and the operational safety and efficiency at nuclear power plants.

Investigating Drivers’ Awareness of Automated Vehicle Modes Using the State Diagram Method

Driving automation introduces multiple driving modes to maximize the system’s benefits, but drivers must monitor and stay aware of these modes, which can sometimes lead to mode confusion. We modified Degani and Heymann’s state diagram method to assess the mode structure of a hypothetical driving automation system and the likelihood of discrepancies between drivers’ mode awareness and the system’s actual mode. We used the modified method and driving simulation data from participants who weren’t fully informed about all automation modes. The modified method visualized all possible combinations of automation modes and drivers’ mode awareness, highlighting where they diverge and estimating the frequency of the divergence. The diagram identified areas where human-machine interface (HMI) design can help drivers maintain accurate mode awareness. The modified state diagram method provides actionable insights for designing HMIs to reduce mode confusion and can be used to develop computational models of automation mode structures.

Human Machine Interface Design for Robotized Road Works in InfraROB project

Human Machine Interface Design for Robotized Road Works in InfraROB project

Assessing driver distraction from in-vehicle information system: an on-road study exploring the effects of input modalities and secondary task types

In-vehicle information system (IVIS) use is prevalent among young adults. However, their interaction with IVIS needs to be better understood. Therefore, an on-road study aims to explore the effects of input modalities and secondary task types on young drivers' secondary task performance, driving performance, and visual glance behavior. A 2 × 4 within-subject design was undertaken. The independent variables are input modalities (auditory-speech and visual-manual) and secondary task types (calls, music, navigation, and radio). The dependent variables include secondary task performance (task completion time, number of errors, and SUS), driving performance (average speed, number of lane departure warnings, and NASA-TLX), and visual glance behavior (average glance duration, number of glances, total glance duration, and number of glances over 1.6 s). The statistical analysis result showed that the main effect of input modalities is significant, with more distraction during visual-manual than auditory-speech. The main impact of secondary task types was also substantial across most metrics, aside from average speed and average glance duration. Navigation and music were the most distracting, followed by calls, and radio came in last. The distracting effect of input modalities is relatively stable and generally not moderated by the secondary task types, except radio tasks. The findings practically benefit the driver-friendly human–machine interface design, preventing IVIS-related distraction.

Effects of various in-vehicle human–machine interfaces on drivers’ takeover performance and gaze pattern in conditionally automated vehicles

With the era of automated driving approaching, designing an effective and suitable human–machine interface (HMI) to present takeover requests (TORs) is critical to ensure driving safety. The present study conducted a simulated driving experiment to explore the effects of three HMIs (instrument panel, head-up display [HUD], and peripheral HMI) on takeover performance, simultaneously considering the TOR type (informative and generic TORs). Drivers’ eye movement data were also collected to investigate how drivers distribute their attention between the HMI and surrounding environment during the takeover process. The results showed that using the peripheral HMI to present TORs can shorten takeover time, and drivers rated this HMI as more useful and satisfactory than conventional HMIs (instrument panel and HUD). Eye movement analysis revealed that the peripheral HMI encourages drivers to spend more time gazing at the road ahead and less time gazing at the TOR information than the instrument panel and HUD, indicating a better gaze pattern for traffic safety. The HUD seemed to have a risk of capturing drivers’ attention, which resulted in an ‘attention tunnel,’ compared to the instrument panel. In addition, informative TORs were associated with better takeover performance and prompted drivers to spend less time gazing at rear-view mirrors than generic TORs. The findings of the present study can provide insights into the design and implementation of in-vehicle HMIs to improve the driving safety of automated vehicles.

Cognitive Principles for Remote Condition Monitoring Applied to a Rail Pantograph System

Remote condition monitoring (RCM) aims to ensure the availability of railway assets. Previous work has indicated the importance of a user-centred RCM design approach based on cognitive principles, but there has been no known demonstration of the application of these principles. The following paper takes this theory-based approach and applies it to the design of an RCM system for the rail pantograph/Overhead Line (OHL) system. The paper first presents a high-level conceptual architecture, based on four stages of cognitive decision-making (notification, acceptance, analysis and clearance), linked to the wider monitoring architecture. Second, the paper uses cognitive principles to propose demonstration Human–Machine Interface designs for the OHL system. These HMIs were presented in an evaluation with subject matter experts. The outcomes of the process generated user-centred design recommendations for RCM. Furthermore, the evaluation suggested the importance of multiple paths through the HMI dependent on the type and urgency of fault. Finally, the outcomes of the evaluation also highlighted the importance of considering context when deploying user-centred RCM.

From Distraction to Action: Elevating Situation Awareness with Visual Assistance in Level 3 Autonomous Driving

This study examines the impact of visual assistance and cognitive load on situation awareness (SA) during takeover events in Level 3 (L3) autonomous driving, where drivers are permitted to engage in Non-Driving Related Tasks (NDRTs). Utilizing a driving simulator, the research explores how different NDRTs, occupying various sensory channels, influence drivers’ SA under two visual assistance conditions: full marking and key marking. Results from the Situation Awareness Global Assessment Technique (SAGAT) and NASA Task Load Index (NASA-TLX) scales, along with eye-tracking data, demonstrate significant interactions between visual assistance conditions and tasks, highlighting key marking’s superior effectiveness in enhancing SA across all NDRTs. The study reveals that cognitive workload, largely arising from NDRTs rather than road attention, remains unaffected by visual assistance conditions. Findings suggest that drivers’ strategies for allocating attention are consistent, regardless of the visual task presence. This research contributes to understanding SA in autonomous driving takeovers, offering valuable insights for developing Human-Machine Interface (HMI) designs to improve SA and takeover performance in autonomous driving systems.

Usability Testing of Novel IoT-Infused Digital Services on Farm Equipment Reveals Farmer’s Requirements towards Future Human–Machine Interface Design Guidelines

The application of digital technologies in the agricultural sector is increasing. One of the new key technologies is the Internet of Things (IoT), which can facilitate the everyday work of farmers. For the successful adoption of IoT-enabled digital products and to ensure improved workflows, the usability of human–machine interfaces is highly important. Various design approaches of human–machine interfaces (HMIs) can currently be found in the monitoring of agricultural machinery. In this work, the most well-known HMIs are considered. Based on a usability test (participants n = 9), the user interface (UI) of a novel mobile application (NEVONEX Cockpit App) was chosen as an example of a design approach of an IoT ecosystem that is oriented towards the UI design of mobile applications. This work aims to identify the weak points of this UI. Conclusions about the needs, and thus an improvement of the user experience, are based on the suggestions for improvement and the information about the general requirements of farmers for a UI for agricultural machinery. Here, it was found that most farmers are satisfied with the UI design of their familiar tractor monitors. According to the survey, the three most important points to be considered in the UI design are intuitive operation and menu navigation, easy access to the essential functions and buttons, and sufficiently large control panels. The conducted usability tests can be considered a basis for developing HMIs for agriculture machinery. Through repeated execution of comparable usability tests, there is the possibility of developing future HMI guidelines for agriculture to improve the user experience (UX). For the NEVONEX ecosystem, feedback from the user interface testing was incorporated in a major revision of the Cockpit App’s design, where a lot more display space was given to the agronomic digital services by smartly arranging infrastructure functions in tiles.

Moving to Advanced Research on Human Machine Interface Design

This poster paper will discuss a variety of items related to Human-Machine Teaming and research in support in-creasing control of autonomous machines present in phys-ical problem domains of interest. Many military tasks can be decomposed into their primary elements – intelligence preparation, reconnaissance, movement, maneuver, fires, and and support across the combat domains of interest – air, land, sea, etc. There are an increasing number of au-tonomous and semi-autonomous ground-based systems available, such as the Multi-Utility Tactical Transport (MUTT) Unmanned Ground Vehicle, for movement of materials, or Quadrupedal Unmanned Ground Vehicle (QUGV) for the disposal of explosive ordinance, comple-mented by aerial platforms considering of a wide variety of Unmanned Aerial Systems (UAS) for the gathering and transmitting of information, held together by a common backbone and network. The preponderance of new systems and capabilities brings new issues, one of which is critical to research - how to control and manage a large number of systems. Commercial systems of significantly less capability, such as light-up drone shows, involve approximately 2 people per 100 drones; meaning that a single controller for a sin-gle drone is simply a non-starter. How can research be applied to scale the complexity of operations upwards without additional demands of personnel? This poster discusses several portions of early-stage research into a variety of applications, including: Reasoning about mixed-team processes, includ-ing real/synthetic teammates, and how infor-mation gathered about the use of synthetic teammates within simulation can be transitioned into utilization for a robotic teammate Embedding affective information into dialogue channels in order to save cognitive bandwidth Repurposing of foundational dialogue models for specific tasks and purposes Utilizing psychological research to design sys-tems in order to infer user intent Theory of mind research for autonomous systems regarding their human operators Simulated environments and agents in order to test the simulations in representative areas. Common-Sense reasoning augmented by Large Language Model technology for instructing ro-botic platforms The following things will be discussed at the poster ses-sion, representing a portfolio of ongoing work addressing the problems of large-scale multi-agent command and control. While the near-term application of these tech-nologies is into military problem domains; the poster presentation is cleared for public release.

The effect of human-machine interface modality, specificity, and timing on driver performance and behavior while using vehicle automation

The effectiveness of the human–machine interface (HMI) in a driving automation system during takeover situations is based, in part, on its design. Past research has indicated that modality, specificity, and timing of the HMI have an impact on driver behavior. The objective of this study was to examine the effectiveness of two HMIs, which vary by modality, specificity, and timing, on drivers’ takeover time, performance, and eye glance behavior. Drivers’ behavior was examined in a driving simulator study with different levels of automation, varying traffic conditions, and while completing a non-driving related task. Results indicated that HMI type had a statistically significant effect on velocity and off-road eye glances such that those who were exposed to an HMI that gave multimodal warnings with greater specificity exhibited better performance. There were no effects of HMI on acceleration, lane position, or other eye glance metrics (e.g., on road glance duration). Future work should disentangle HMI design further to determine exactly which aspects of design yield between safety critical behavior.

Navigating from data-driven design to designing with ML: a case study of truck HMI system design

AbstractData-driven design is believed to be empowered by machine learning (ML) with advanced pattern classification and prediction. However, research on how ML can be used to support automotive human-machine interface (HMI) design is lacking. We presented a case study of truck HMI design to understand the current data use and expectations of ML in the design process. Findings show decentralized data practices, the role of expertise in decision-making, and the envisioned reactive use of ML, where we underscore the implications for advancing human-ML collaboration in designing future truck HMI systems.

The Impact of Transparency on Driver Trust and Reliance in Highly Automated Driving: Presenting Appropriate Transparency in Automotive HMI

Automation transparency offers a promising way for users to understand the uncertainty of automated driving systems (ADS) and to calibrate their trust in them. However, not all levels of information may be necessary to achieve transparency. In this study, we conceptualized the transparency of the automotive human–machine interfaces (HMIs) in three levels, using driving scenarios comprised of two degrees of urgency to evaluate drivers’ trust and reliance on a highly automated driving system. The dependent measures included non-driving related task (NDRT) performance and visual attention, before and after viewing the interface, along with the drivers’ takeover performance, subjective trust, and workload. The results of the simulated experiment indicated that participants interacting with an SAT level 1 + 3 (system’s action and projection) and level 1 + 2 + 3 (system’s action, reasoning, and projection) HMI trusted and relied on the ADS more than did those using the baseline SAT level 1 (system’s action) HMI. The low-urgency scenario was associated with higher trust and reliance, and the drivers’ visual attention and NDRT performance improved after viewing the HMI, but not statistically significantly. The findings verified the positive role of the SAT model regarding human trust in the ADS, especially in regards to projection information in time-sensitive situations, and these results have implications for the design of automotive HMIs based on the SAT model to facilitate the human–ADS relationship.

디지털 적층 제조 장비의 사용자 중심 HMI 디자인 가이드 제안

With the advancement of metal additive manufacturing technology and the digitization of equipment, the need for user-centered design in HMI (Human-Machine Interface) has become increasingly apparent. However, research on design guides and usability test for industrial equipment has been lacking. Therefore, this study applied user-centered design principles to design and manufacture a prototype HMI for metal additive manufacturing equipment, followed by a usability test of the HMI prototype. Based on usability testing and in-depth interview analysis, this study proposes UX/UI design guidelines for user-centered industrial equipment HMI. The usability evaluation highlighted the need to improve the visibility of the system's real-time status, enhance interface intuitiveness for error prevention, and address emergency situations. It is crucial that the interface reflects both the visible and invisible states of the equipment, considering that the operation targets are mechanical equipment but the operating devices are HMI screens. Furthermore, for industrial equipment with numerous complex functions, it is essential to restrict or guide user actions through permission settings. Through the industrial PBF (Powder Bed Fusion) method metal additive manufacturing equipment HMI design guides proposed by this study, we anticipate the development of user-friendly and productivity-enhancing HMI designs.

Exploring factors in Human Cognition that affect Human Performance during Takeover in Autonomous Driving

Driver's mental load is an important influence on the design of the human-machine interface (HMI) for self-driving cars. During the autopilot-forced takeover process, aspects of the user's cognition, decision-making, attention, and emotional state affect the takeover performance. Therefore, the cognitive and psychological states of drivers must be considered in the HMI design of self-driving cars. And a good HMI design will reduce the user's mental burden and improve the takeover performance. In this study, a model affecting drivers' takeover performance was designed through human factors. And we refine the analysis from three aspects of cognitive load, emotional load, and environmental load by making subjective likert-style-questionnaires, derive the main influencing factors through the reliability analysis of the data, factor analysis, and other methods. This work finds that the clarity, comprehensiveness, and level of humanization of takeover messages, as well as the driver's arousal level when receiving the message, have the greatest impact on the driver's workload.

Designing for Trust: How Human-Machine Interface Can Shape the Future of Urban Air Mobility

This study investigates the influence of human-machine interface (HMI) design on passenger trust in autonomous electric vertical take-off and landing (eVTOL) vehicles. An immersive simulator-based experiment was conducted with 34 participants, exposing them to four HMI conditions: baseline, movement, hazard detection, and full information condition. As related measures of passenger trust, we collected self-reported measures including trust, perceived safety, perceived reliability, and intention to use. In addition, physiological measures including gaze behavior, electrodermal activity, and heart rate were collected. The results indicated that movement and hazard detection information improved passenger trust, suggesting that HMI design could play a crucial role in enhancing the acceptance of autonomous eVTOLs. In addition, gaze behavior showed a stronger relationship with self-reported trust than other physiological measures. The findings underscore the importance of HMI design in fostering passenger trust, which is critical to the success of urban air mobility.

Methodology based on machine learning through neck motion and POF-based pressure sensors for wheelchair operation

Polymer Optical Fiber (POF)-based sensors have gained recognition in recent years for biomedical applications because of their low cost, physical properties, and feasibility. A novel methodology is proposed here for classifying neck movements using POF- based pressure sensors and machine learning algorithms. To address this, signal pre-processing, feature extraction, and selection methods are implemented, considering variance, root mean square, and Hjorth parameters. Linear Discriminant Analysis, Support Vector Machine, k-Nearest Neighbors (kNN), and Decision Tree (DT) were used for classification. A maximum accuracy of 0.91 was obtained with kNN and DT for recognizing four neck movements by using the best discriminant nine features. These findings indicate that the proposed methodology is suitable for neck-motion classification using POF-based pressure sensors. Future work will focus on the implementation of this strategy for the design of intelligent Human Machine Interfaces based on electric-powered wheelchairs, which would allow for more independence for people with upper- and lower-limb disabilities.

Effects of Control Transition Strategies and Human-Machine Interface Designs on Driver Performance in Automated Driving Systems

Effects of Control Transition Strategies and Human-Machine Interface Designs on Driver Performance in Automated Driving Systems

Requirements for Human-Machine-Interaction Applications in Production and Logistics within Industry 5.0 – A Case Study Approach

This research paper examines the evolution of human-machine interfaces (HMIs) in the manufacturing and logistics sectors in the context of Industry 5.0 (I5.0). I5.0 introduces a human-centered paradigm, and as digitalization transforms processes in these domains, human-machine interaction is undergoing a notable shift. Yet, there is a lack of I5.0 HMI requirements and design guidelines. Through a literature review, the research identifies and investigates general requirements for HMIs in I5.0. Findings include ethical principles, ergonomics, adaptability, human-centered design, intuitive interaction, and effective collaboration. It also uses a case study approach to highlight potential strategies and discuss challenges. The results provide valuable insights into considerations for designing effective HMIs for I5.0 applications, enhancing the user experience, improving productivity, and facilitating successful human-machine interactions.