Human-machine Interface Concepts Research Articles

To what extent implanting single vs pairs of magnets per muscle affect the localization accuracy of the myokinetic control interface? Evidence from a simulated environment.

We recently proposed a new concept of human-machine interface to control hand prostheses which we dubbed the myokinetic control interface. Such interface detects muscle displacement during contraction by localizing permanent magnets implanted in the residual muscles. So far, we evaluated the feasibility of implanting one magnet per muscle and monitoring its displacement relative to its initial position. However, multiple magnets could actually be implanted in each muscle, as using their relative distance as a measure of muscle contraction could improve the system robustness against environmental disturbances. Here, we simulated the implant of pairs of magnets in each muscle and we compared the localization accuracy of such system with the one magnet per muscle approach, considering first a planar and then an anatomically appropriate configuration. Such comparison was also performed when simulating different grades of mechanical disturbances applied to the system (i.e. shift of the sensor grid). We found that implanting one magnet per muscle always led to lower localization errors under ideal conditions (i.e. no external disturbances). Differently, when mechanical disturbances were applied, magnet pairs outperformed the single magnet approach, confirming that differential measurements are able to reject common mode disturbances. We identified important factors affecting the choice of the number of magnets to implant in a muscle. Our results provide important guidelines for the design of disturbance rejection strategies and for the development of the myokinetic control interface, as well as for a whole range of biomedical applications involving magnetic tracking.

Adopting an HMI for overtaking assistance - Impact of distance display, advice, and guidance information on driver gaze and performance

Conveying supporting information by an in-vehicle human–machine interface (HMI) overtaking assistance system represents a promising approach to facilitate safe overtaking maneuvers. Although various researchers have attempted to examine the safety of systems, numerous HMI concepts are not validated and the impacts of systems on driver behavior following exposure to a connected environment remain unclear. Thus, the objective of this study is to assess the effectiveness of various HMI concepts and their effects on drivers' gaze behavior and driving performance in the connected environment. Three types of supporting information, namely distance, advice and guidance, were designed based on implicit and explicit concepts and displayed on the HMI. The guidance information provides more detailed maneuver instructions. A driving simulator experiment was conducted to imitate a connected environment and included an overtaking scenario on a two-lane highway under three distance gaps (short, middle and long) of oncoming vehicles and four HMI conditions including baseline (3*4). 39 participants (mean = 28.3 years, SD = 8.2 years) drove the simulator. Visual attention, driving maneuver, and subjective evaluation data were collected. The results indicated that the supporting information was valid in facilitating overtaking behavior compared without HMI, particularly under the middle distance gap scenario. Advice information was the most effective, required the least attention, exhibited the best performance, and was accepted by drivers. Detailed maneuver instructions were perceived as helpful, although no effect on acceptance was found. The findings of the study provide valuable insights into the development of user-friendly overtaking assistance systems.

Minimal Risk Maneuvers of Automated Vehicles: Effects of a Contact Analog Head-Up Display Supporting Driver Decisions and Actions in Transition Phases

Minimal risk maneuvers (MRMs), as part of highly automated systems, aim at minimizing the risk during a transition phase from automated to manual driving. Previous studies show that many drivers have an urge to intervene in transition phases despite the system’s capability to safely come to a standstill. A human–machine interface (HMI) concept was developed to support driver decisions by providing environmental information and action recommendations. This was investigated in a static driving simulator experiment with 36 participants. Two scenarios that differed in the traffic on the adjacent left lane were implemented and the HMI concept displayed the content accordingly. Results of the study again show a high intervention rate of drivers overtaking the obstacle from the left, even if the lane is occupied by other vehicles. The HMI concept had a positive influence on the manner of intervention by encouraging a standstill in the shoulder lane. Nevertheless, negative consequences included accidents and dangerous situations, but at lower frequencies and proportions during drives with the HMI concept. In conclusion, the risk during the transition phase was reduced. Furthermore, the results showed a significant decrease in the subjective workload and a positive influence on the drivers’ understanding and predictability of the automated system.

A toolkit to co-develop Human-Machine Interfaces together with users in all automated transport modes

Involving users in the process of developing future mobility solutions is a key requirement in research and innovation. In the project Drive2theFuture, we investigate how user acceptance of connected, cooperative and automated transport can be achieved for all stakeholders. To investigate and understand user behavior for automated transport solutions, we propose a software toolkit to co-develop Human-Machine Interfaces (HMI) together with users. It enables HMI researchers to rapidly prototype, experience and evaluate novel HMI concepts for the transport modes road, rail, aviation and maritime by providing a library of HMI elements and recommendations, features for HMI personalization and a model to predict user's emotional state. Its application is suited for the development and experience of concepts in 2D or Virtual Reality (VR).

Innovative HMI and Control Concept for Efficient Thermal Management of Electric Vehicles

This study presents a practical design and implementation of a Thermal Management (TM) concept for Battery-Electric Vehicles (BEVs) including a novel Human-Machine Interface (HMI). The developed system uses a combination of convective air conditioning and radiation heating to cool down and heat up the passengers efficiently. The proposed contribution can be subdivided into four parts: 1) description of the Heating, Ventilation and Air Conditioning (HVAC) system; 2) development of a novel HMI; 3) derivation of an appropriate operating strategy concept; 4) validation of the entire TM system based on a user study using a demonstrator vehicle. The results show that the developed TM concept can substantially support the user to operate the HVAC system more efficiently. A user study was conducted to evaluate passenger comfort and usability. The study revealed that users tend to operate a conventional input panel of an HVAC system inefficiently in some cases. They could achieve better task performance with the new HMI proposed in this study. However, the acceptance of the new concept was rated lower for the new system, as users were already accustomed to the conventional input panel design.

How to Display Vehicle Information to Users of Automated Vehicles When Conducting Non-Driving-Related Activities

Automated vehicles (AVs) are expected to enable users to engage in non-driving-related activities (NDRAs). However, users do not easily trust an automated vehicle which poses new challenges for automotive human-machine interfaces (HMIs). Over-presenting vehicle information can distract users from NDRAs, and under-presentation can impact trust and user experience (UX) negatively. To investigate how to best present vehicle information to foster users' trust and UX while performing NDRAs, we designed two in-vehicle HMI concepts: 1) A colored and animated light bar display around the windshield and 2) a windshield display interface presenting pictograms and numbers. Results from a simulator study ($N=18$) indicate that both concepts contribute to a high trust level and UX while not affecting the NDRA performance compared to the baseline of not showing vehicle information. In addition, the light bar provides better UX than the windshield display and is also preferred by users. With our findings, we contribute to the effective design of presenting vehicle information in automated vehicles.

How to Keep Drivers Attentive during Level 2 Automation? Development and Evaluation of an HMI Concept Using Affective Elements and Message Framing

With Level 3 and 4 automated driving activated, users will be allowed to engage in a wide range of non-driving related activities (NDRAs). Although Level 2 automation can appear very similar to L3 and L4, drivers are required to always monitor the system. However, past research has found drivers neglect this obligation at least partly and instead engage in NDRAs. Since this behavior can have negative impacts on traffic safety, the goal of this work was to develop a human–machine interface (HMI) concept to motivate users to continue their supervision task. This work’s concept used message framing in connection with affective elements. Every three minutes, messages were displayed on the head-up display. To evaluate the affective message concept’s (AMC) effectiveness, we conducted a between-subject driving simulator study (baseline vs. advanced HMI) with 32 participants and 45 min of driving time with both L2 and L4 phases and a silent system malfunction. Results show the road attention ratio decreases and the NDRA engagement ratio increases over time only for baseline participants. Participants supported by the AMC did not show a change over time in monitoring behavior and NDRA engagement. However, no effect on the drivers’ reaction to the system failure became apparent. No effects on subjective workload and user experience were found. Additional research is needed to further investigate the safety implications and long-term effectiveness of the concept, as well as a driver-state-dependent design.

Transport Automation in Urban Mobility: A Case Study of an Autonomous Parking System

Parking road vehicles is one of the most tedious and challenging tasks a human driver performs. Despite the low speeds involved, parking manoeuvres are among the main causes of minor and sometimes major traffic accidents, especially in urban areas where limited parking spaces are available. Furthermore, searching for a parking space wastes time and contributes to unnecessary road occupancy and pollution. This paper is dedicated to the development of an autonomous parking system for on-street parking in urban areas. The system is capable of fully automated parking manoeuvres from drop-off to pick-up zones, thus removing human drivers from the vehicle control loop. The system autonomously navigates to the parking space and parks the vehicle without human intervention. The proposed system incorporates a communication protocol that connects automated vehicles, parking infrastructure, and drivers. Several convenient human–machine interface concepts for efficient system communication and state monitoring have been developed. A methodology for validating the system in real time is proposed, which includes functionality requirements and a description of parallel and perpendicular parking manoeuvres. The proposed pipeline is tested on an electric vehicle platform with automated functions, where successful technological functionality is demonstrated.

Data-Driven Real-Time Magnetic Tracking Applied to Myokinetic Interfaces

A new concept of human-machine interface to control hand prostheses based on displacements of multiple magnets implanted in the limb residual muscles, the myokinetic control interface, has been recently proposed. In previous works, magnets localization has been achieved following an optimization procedure to find an approximate solution to an analytical model. To simplify and speed up the localization problem, here we employ machine learning models, namely linear and radial basis functions artificial neural networks, which can translate measured magnetic information to desired commands for active prosthetic devices. They were developed offline and then implemented on field-programmable gate arrays using customized floating-point operators. We optimized computational precision, execution time, hardware, and energy consumption, as they are essential features in the context of wearable devices. When used to track a single magnet in a mockup of the human forearm, the proposed data-driven strategy achieved a tracking accuracy of 720 μm 95% of the time and latency of 12.07 μs. The proposed system architecture is expected to be more power-efficient compared to previous solutions. The outcomes of this work encourage further research on improving the devised methods to deal with multiple magnets simultaneously.

Looking at HMI Concepts for Highly Automated Vehicles: Permanent vs. Context-Adaptive Information Presentation

Abstract To facilitate the usage and expected benefits of higher-level automated vehicles, passengers’ distrust and safety concerns should be reduced through increasing system transparency (ST) by providing driving-related information. We therefore examined the effects of ST on passengers’ gaze behavior during driving, trust in automated driving and evaluation of different human-machine interface (HMI) concepts. In a driving simulator, 50 participants experienced three identical highly automated drives under three HMI conditions: no HMI (only conventional speedometer), context-adaptive HMI (all system information only available in more complex situations) or permanent HMI (all system information permanently available). Compared to driving without HMI, the introduction of the two HMIs resulted in significantly higher usage of the center stack display (i.e. gazes towards the HMIs), which was accompanied by significantly higher trust ratings. The considerable differences in information availability provided by the context-adaptive versus permanent HMI did not reflect in similarly considerable differences regarding the passengers’ gaze behavior or accompanied trust ratings. Additionally, user experience evaluations expressed preferences for the context-adaptive HMI. Hence, the permanent HMI did not seem to create benefits over the context-adaptive HMI, supporting the usage of more economical, context-adaptive HMIs in higher-level automated vehicles.

Improving Passenger Experience and Trust in Automated Vehicles Through User-Adaptive HMIs: “The More the Better” Does Not Apply to Everyone

Automated vehicles promise transformational benefits for future mobility systems, but only if they will be used regularly. However, due to the associated loss of control and fundamental change of in-vehicle user experience (shifting from active driver to passive passenger experience), many humans have reservations toward driving automation, which question their sufficient usage and market penetration. These reservations vary based on individual characteristics such as initial attitudes. User-adaptive in-vehicle Human-Machine Interfaces (HMIs) meeting varying user requirements may represent an important component of higher-level automated vehicles providing a pleasant and trustworthy passenger experience despite these barriers. In a driving simulator study, we evaluated the effects of two HMI versions (with permanent vs. context-adaptive information availability) on the passenger experience (perceived safety, understanding of driving behavior, driving comfort, driving enjoyment) and trust in automated vehicles of 50 first-time users with varying initial trust (lower vs. higher trust group). Additionally, we compared the user experience of both HMIs. Presenting driving-related information via HMI during driving improved all assessed aspects of passenger experience and trust. The higher trust group experienced automated driving as safest, most understandable and most comfortable with the context-adaptive HMI, while the lower trust group tended to experience the highest safety, understanding and comfort with the permanent HMI. Both HMIs received positive user experience ratings. The context-adaptive HMI received generally more positive ratings, even though this preference was more pronounced for the higher trust group. The results demonstrate the potential of increasing the system transparency of higher-level automated vehicles through HMIs to enhance users’ passenger experience and trust. They also consolidate previous findings on varying user requirements based on individual characteristics. User group-specific HMI effects on passenger experience support the relevance of user-adaptive HMI concepts addressing varying needs of different users by customizing HMI features, such as information availability. Consequently, providing full information permanently cannot be recommended as a universal standard for HMIs in automated vehicles. These insights represent next steps toward a pleasant and trustworthy passenger experience in higher-level automated vehicles for everyone, and support their market acceptance and thus the realization of their expected benefits for future mobility and society.

Human-Centered Design for an In-Vehicle Truck Driver Fatigue and Distraction Warning System

Driver fatigue and distraction are major road safety issues globally; developing driver state detection and warning technology to help reduce impairment-related incidents is a promising approach. The aim of this case study was to design an effective Human Machine Interface (HMI) for a near-market driver warning system primarily aimed at commercial truck driving. A human-centered design (HCD) process was employed for the development and evaluation. Application of HCD here was a multi-stage iterative process: a comprehensive literature review; developing a context of use description; undertaking truck driver interviews; identifying user needs and associated design requirements; conducting two design workshops; operationalizing the design; running HMI evaluation studies, and finalizing the HMI concepts. As a result of the iterative HCD process, the HMI comprises a multi-modal warning system (visual, auditory and tactile) with two levels for driver fatigue and an escalating system for driver distraction. Following this successful HCD process, further on-road evaluation work is proposed before the final version of the HMI is manufactured.

UUX Evaluation of a Digitally Advanced Human–Machine Interface for Excavators

With the evaluation of a next-generation human–machine interface (HMI) concept for excavators, this study aims to discuss the HMI quality measurement based on usability and user experience (UUX) metrics. Regarding the digital transformation of construction sites, future work environments will have to be capable of presenting various complex visual data and enabling efficient and safe interactivity while working. The evaluated HMI focused on introducing a touch display-based interface, providing advanced operation functions and different interaction modalities. The assessment of UUX should show whether the novel HMI can be utilised to perform typical tasks (usability) and how it is accepted and assessed in terms of non-instrumental qualities (user experience, UX). Using the collected data, this article also aims to contribute to the general discussion about the role of UX beyond usability in industrial applications and deepen the understanding of non-instrumental qualities when it comes to user-oriented process and machine design. The exploratory study examines insights into the application of elaborated UUX measuring tools like the User Experience Questionnaire (UEQ) on the interaction with industrial goods accompanied by their rating with other tools, namely System Usability Scale (SUS), Intuitive Interaction Questionnaire (INTUI) and the National Aeronautics and Space Administration (NASA) Task Load Index (NASA-TLX). Four goals are pursued in this study. The first goal is to compare in-depth two different ways of interaction with the novel HMI—namely one by a control pad on the right joystick and one by touch. Therefore, a sample of 17 subjects in total was split into two groups and differences in UUX measures were tested. Secondly, the performances of both groups were tested over the course of trials to investigate possible differences in detail. The third goal is to interpret measures of usability and user experience against existing benchmark values. Fourth and finally, we use the data gathered to analyse correlations between measures of UUX. The results of our study show that the different ways of interaction did not impact any of the measures taken. In terms of detailed performance analysis, both groups yielded differences in terms of time per action, but not between the groups. The comparison of UUX measures with benchmark values yielded mixed results. The UUX measures show some relevant significant correlations. The participants mostly reported enjoying the use of the HMI concept, but several practical issues (e.g., efficiency) still need to be overcome. Once again, the study confirms the urge of user inclusion in product development. Especially in the course of digitalisation, as big scale advancements of systems and user interfaces bring uncertainty for many manufacturers regarding whether or how a feature should be integrated.

Unconstrained design: improving multitasking with in-vehicle information systems through enhanced situation awareness

ABSTRACTIn the age of information, in-vehicle multitasking is inevitable. The popularity of the automobile, in combination with the present information age, create a growing demand to do more in-vehicle than simply focus on the road. Unconstrained Design, a philosophy which supports rather than constrains multitasking, is proposed as a path toward enhancing performance in-vehicle. Situation Awareness (SA), a theory allowing designers to understand how operators interact in dynamic, complex environments, is used to frame this experimental investigation. Two SA-grounded human-machine interface concepts are proposed, designed to support drivers to multitask in-vehicle when frequent task switching is required. The first focuses upon supporting preparation for a Non-Driving Related Activity (NDRA), and the second upon supporting the Driving Related Activity (DRA) when an NDRA is active. While multitasking, Contextual Cueing, using a Head-up Display, produced significant reductions in NDRA response time, while an auditory lane keeping aid increased the amount of time a driver spent in the central region of a lane. The combined evidence suggests that using SA and Unconstrained Design to create of IVIS that support drivers’ ability to multitask in-vehicle can lead to task performance improvements.

Takeover Requests in Highly Automated Truck Driving: How Do the Amount and Type of Additional Information Influence the Driver–Automation Interaction?

Vehicle automation is linked to various benefits, such as increase in fuel and transport efficiency as well as increase in driving comfort. However, automation also comes with a variety of possible downsides, e.g., loss of situational awareness, loss of skills, and inappropriate trust levels regarding system functionality. Drawbacks differ at different automation levels. As highly automated driving (HAD, level 3) requires the driver to take over the driving task in critical situations within a limited period of time, the need for an appropriate human–machine interface (HMI) arises. To foster adequate and efficient human–machine interaction, this contribution presents a user-centered, iterative approach for HMI evaluation of highly automated truck driving. For HMI evaluation, a driving simulator study [n = 32] using a dynamic truck driving simulator was conducted to let users experience the HMI in a semi-real driving context. Participants rated three HMI concepts, differing in their informational content for HAD regarding acceptance, workload, user experience, and controllability. Results showed that all three HMI concepts achieved good to very good results in these measures. Overall, HMI concepts offering more information to the driver about the HAD system showed significantly higher ratings, depicting the positive effect of additional information on the driver–automation interaction.

Development and usability of a novel holographic 3-dimensional user interface using hand-gesture control for a future vehicle application by older adults

The Deep Orange program immerses automotive engineering students into the world of an OEM as part of their graduate education. While developing the program’s seventh vehicle concept, students explored new human machine interface concepts with the goal of having a clean, minimal interior design. One outcome of their ideation process is a concept for a holographic companion that can act as a concierge for all functions of the vehicle. After creating a prototype of the holographic display using existing technologies and developing a user interface controlled by hand gestures, a usability study was performed with older adults. The results suggest the system was not intuitive. Participants demonstrated better performance with tasks using discrete hand motions in comparison to those that required continuous motions. The data were helpful to understand the challenges of untrained users interacting with a new HMI system.

Strategies to Assist Drivers in Remaining Attentive While Under Partially Automated Driving: Verification of Human–Machine Interface Concepts

Visual inattention is a major concern with partially automated driving systems that assume vehicle steering functions. The safety concept for these systems capable of maintaining vehicle lane position and headway depends on an attentive driver who detects and appropriately responds to objects and events that are beyond the limitations of the sensors. Embedded within a proposed system are features designed to help drivers perform these functions and remain attentive to the driving environment. This study served to validate human–machine interface (HMI) strategies and concepts that can be applied to production-intent partially automated system HMI to achieve intended safety goals in assisting drivers to maintain attention to driving. Previous validation efforts, conducted as part of a NHTSA study, proved insufficient for verification of these driver performance functions since they relied on an incomplete version of alternative HMI concepts. The current study, with a sample of 25 licensed drivers, used approaches that more closely approximated the potential production-intent partially automated HMI concepts, including the introduction of consequences for failing to respond to alerts. Work was performed by using an advanced prototype capable of mimicking the basic functions afforded in a partially automated system. Driver responses to unexpected lane drift events were also examined. Results found that HMI concepts that introduce consequences for driver nonresponse situations substantially increase driver compliance to system cues, prompts, and alerts. Results of this study suggest that the potential production design partially automated system HMI concepts can assist drivers in maintaining their attention to driving.

Evaluation of speech-based HMI concepts for information exchange tasks: A driving simulator study

Due to the mobile Internet revolution, people tend to browse the Web while driving their car which puts the driver's safety at risk. Therefore, an intuitive and non-distractive in-car speech interface to the Web needs to be developed. Before developing a new speech dialog system (SDS) in a new domain developers have to examine the user's preferred interaction style and its influence on driving safety.This paper reports a driving simulation study, which was conducted to compare different speech-based in-car human–machine interface concepts concerning usability and driver distraction. The applied SDS prototypes were developed to perform an online hotel booking by speech while driving. The speech dialog prototypes were based on different speech dialog strategies: a command-based and a conversational dialog. Different graphical user interface (GUI) concepts (one including a human-like avatar) were designed in order to support the respective dialog strategy the most and to evaluate the effect of the GUI on usability and driver distraction.The results show that only few differences concerning speech dialog quality were found when comparing the speech dialog strategies. The command-based dialog was slightly better accepted than the conversational dialog, which seems to be due to the high concept error rate of the conversational dialog. A SDS without GUI also seems to be feasible for the driving environment and was accepted by the users. The comparison of speech dialog strategies did not reveal differences in driver distraction. However, the use of a GUI impaired the driving performance and increased gaze-based distraction. The presence of an avatar was not appreciated by participants and did not affect the dialog performance. Concerning driver distraction, the virtual agent did neither negatively affect the driving performance nor increase visual distraction.The results implicate that in-car SDS developers should take both speaking styles into consideration when designing an SDS for information exchange tasks. Furthermore, developers have to consider reducing the content presented on the screen in order to reduce driver distraction. A human-like avatar was not appreciated by users while driving. Research should further investigate if other kinds of avatars might achieve different results.

Driver with Varied Disability Level – Vehicle System: New Design Concept, Construction and Standardization of Interfaces

This paper presents an original concept of an urban transport system based on a hybrid vehicle that can move as an electric vehicle driven by a man or as special PodCar vehicle (Rights of Way a; b or c). The system was developed at the Warsaw University of Technology and will be hereinafter referred to as HVTSUA system (Hybrid Vehicle and Transit System for Urban Application). The vehicle in the proposed urban transport system is designed for both able-bodied drivers and passengers, as well as persons with reduced mobility caused by broadly understood age-related diseases or civilization diseases (e.g. victims of accidents, multiple sclerosis, etc.). In this context, an original concept of design and HMI (Human Machine Interface) standardization was presented for this type of vehicle. The concept of HVTSUA builds on the work initiated within the framework of “Eco-Mobility” project co-financed by the European Union. HVTSUA integrates the PodCar system developed there with “Eco-Car” system. The integration of these two elements and equipping them with new technology became the basis for a system with new quality.

A heterogeneous framework for real-time decoding of bioacoustic signals: Applications to assistive interfaces and prosthesis control

Tongue-movement ear pressure (TMEP) signals provide an unobtrusive, completely non-invasive, wearable and assistive human–machine interface (HMI). The HMI concept is based on monitoring volitional bioacoustic activity generated through prescribed tongue motions. In this paper, a heterogeneous decoding framework is presented, enabling effective real-time polychotomous classification between the various tongue actions and dichotomous discrimination of unintended bioacoustic activity from the volitional signals. Using this customised framework and developed software, the real-time performance was evaluated for both three (six subjects) and four action (four subjects) discrimination, using healthy subjects. Ignoring false negative rejections, the system achieved sensitivities of >90% for three-action discrimination and >80% for four action discrimination, across all tested subjects. The interference rejection (IR) capabilities of the framework were also fully demonstrated, using challenging offline data sets. This included a subset of low frequency interference signals with similar temporal characteristics and frequency distributions as the volitional tongue activity. The IR subsystem achieved an average specificity of 76.2% during three-action discrimination and 79.9% during four-action discrimination. To highlight the potential of the system for substituting or augmenting existing assistive interfaces, a case study is presented demonstrating the utility of TMEP signals for hand prosthesis control. Full tongue control was evaluated against three alternative control strategies, namely natural human-hand manipulation, proportional-based control and a hybrid strategy, when performing an everyday object manipulation task. In all cases, the task was completed with the hybrid strategy performing comparably and even outperforming the proportional-based control strategy.