In-vehicle Tasks Research Articles

When Do Users Prefer Voice Control Systems in Vehicles? A Survey of Chinese Drivers

Voice control systems (VCSs) are becoming increasingly common in modern vehicles, but concerns are still expressed by drivers about adopting them in smart cockpits. Previous research on privacy and recognition accuracy may not fully address users’ needs with regard to VCSs. In addition, studies on the safety impacts of different interaction modalities may not reveal drivers’ preference for interaction modalities, given that driver choice may not be solely based on interaction safety. Thus, to better understand and optimize VCSs design, an online survey was conducted in China to investigate drivers’ choices between VCSs and manual interaction. In total, we analyzed 168 drivers’ preferred interaction modality (i.e., either VCSs or manual interaction) in different scenarios defined by passenger presence (i.e., present versus absent), traffic complexity (i.e., low versus high traffic density), roadway type (i.e., urban road versus highway), and task characteristics (i.e., six in-vehicle interaction tasks). Compared with manual interactions, drivers preferred VCSs when driving alone, driving on highways, and driving in complex traffic. Further, drivers preferred VCSs when in-vehicle tasks were more distracting and time-demanding (e.g., compiling a text message). The preference for interaction modalities could also be affected by the characteristics of drivers: those with lower driving frequencies and higher VCSs familiarity and those who perceived higher VCSs usability tended to choose VCSs as an interaction modality for in-vehicle interaction tasks. These findings offer insights for adaptive interface design and future optimization of VCSs.

A comparative analysis of voluntary in-vehicle distractions of drivers' visual attention

Distractions to the driving process, whether inside or outside of the vehicle, are becoming an increasing problem, especially distractions that are voluntary in nature which are avoidable but seem to be prevalent on our streets. A research gap was identified in the lack of comparability of in-vehicle distraction influences on drivers' visual attention. The paper presents research on the effects of in-vehicle distractions on drivers' visual attention, which was performed with 23 participants in a real environment (test track) during normal traffic conditions. The drivers were subjected to 15 in-vehicle distractions, including mobile phone use (handheld and hands-free), finding items inside the vehicle, and controlling and managing various in-vehicle systems. All distractions caused a significant reduction in visual attention to driving-related areas, including tasks such as setting the climate control or radio frequency, which are generally not perceived as disruptions. In fact, setting the climate control, navigational system, and radio frequency diverted visual attention for more than 40 % of the whole task duration. All tasks caused a reduction in the variability of gaze direction onto the windshield, indicating an occurrence of vision tunnelling, which was again very prominent even with tasks that are usually not perceived as very distracting, such as adjusting the side mirrors. The results, therefore, show that in-vehicle tasks that are voluntary in nature may significantly affect drivers' visual attention, and this is not limited to mobile phone and entertainment system use.

Dynamic Selection Slicing-Based Offloading Algorithm for In-Vehicle Tasks in Mobile Edge Computing

With the surge in tasks for in-vehicle terminals, the resulting network congestion and time delay cannot meet the service needs of users. Offloading algorithms are introduced to handle vehicular tasks, which will greatly improve the above problems. In this paper, the dependencies of vehicular tasks are represented as directed acyclic graphs, and network slices are integrated within the edge server. The Dynamic Selection Slicing-based Offloading Algorithm for in-vehicle tasks in MEC (DSSO) is proposed. First, a computational offloading model for vehicular tasks is established based on available resources, wireless channel state, and vehicle loading level. Second, the solution of the model is transformed into a Markov decision process, and the combination of the DQN algorithm and Dueling Network from deep reinforcement learning is used to select the appropriate slices and dynamically update the optimal offloading strategy for in-vehicle tasks in the effective interval. Finally, an experimental environment is set up to compare the DSSO algorithm with LOCAL, MINCO, and DJROM, the results show that the system energy consumption of DSSO algorithm resources is reduced by 10.31%, the time latency is decreased by 22.75%, and the ratio of dropped tasks is decreased by 28.71%.

Impact of In-Air Gestures on In-Car Task’s Diver Distraction

As in-vehicle information systems (IVIS) grow increasingly complex, the demand for innovative artificial intelligence-based interaction methods that enhance cybersecurity becomes more crucial. In-air gestures offer a promising solution due to their intuitiveness and individual uniqueness, potentially improving security in human–computer interactions. However, the impact of in-air gestures on driver distraction during in-vehicle tasks and the scarcity of skeleton-based in-air gesture recognition methods in IVIS remain largely unexplored. To address these challenges, we developed a skeleton-based framework specifically tailored for IVIS that recognizes in-air gestures, classifying them as static or dynamic. Our gesture model, tested on the large-scale AUTSL dataset, demonstrates accuracy comparable to state-of-the-art methods and increased efficiency on mobile devices. In comparative experiments between in-air gestures and touch interactions within a driving simulation environment, we established an evaluation system to assess the driver’s attention level during driving. Our findings indicate that in-air gestures provide a more efficient and less distracting interaction solution for IVIS in multi-goal driving environments, significantly improving driving performance by 65%. The proposed framework can serve as a valuable tool for designing future in-air gesture-based interfaces for IVIS, contributing to enhanced cybersecurity.

A back adjustment based dependent task offloading scheduling algorithm with fairness constraints in VEC networks

In Vehicular Edge Computing (VEC) networks, task offloading scheduling has been drawing more and more attention as an effective way to relieve the computational burden of vehicles. However, with the intelligent and networked development of vehicles, the complex data dependency between in-vehicle tasks brings challenges to offloading scheduling. Moreover, scheduling fairness has a growing impact on the average Quality of Service (QoS) of vehicles in the network. To this end, we propose a dependent task offloading scheduling algorithm with fairness constraints based on a back adjustment mechanism. First, to solve the execution constraint problem caused by dependent tasks and the scheduling fairness problem in multi-user scenarios, a two-level task sorting algorithm is given to determine the scheduling sequence of tasks. Then, the sequential task offloading scheduling process is modeled as a Markov Decision Process (MDP) and solved by a reinforcement learning method. Finally, a back adjustment mechanism is designed to resort the task sequence and achieve the required scheduling fairness by iterative process. The simulation results show that the proposed algorithm significantly improves the scheduling fairness and reduces the average application completion time compared with other algorithms.

Mid-air gestures for in-vehicle media player: elicitation, segmentation, recognition, and eye-tracking testing

There was a problem of great distraction in traditional touch interactive operation of in-vehicle information interface, but the application of gesture recognition technology on in-vehicle system improved this problem. However, users had less experience in this new interactive mode, and the cognitive deviation of the agreement between gestures and commands could directly affect the safety of drivers. The main purpose of this paper was to obtain the user's preference mid-air gestures for the in-vehicle information interface by using the user-elicitation method. In addition, the optimized gesture recognition network structure was applied to the prototype system of in-vehicle information interaction with user-defined mid-air gestures developed by us, and the effectiveness of the system was evaluated by collecting eye movement indicators of users through the simulation of driving eye movement experiments. In the process of user elicitation, the principle of command prompt and agreement rate was introduced. According to the agreement rate elicited by users (AR = .397), we got the gesture consensus set design direction of in-vehicle information control. The experimental results of the eye movement index showed that the method of user-defined mid-air gestures can effectively improve driving safety on in-vehicle media control, and reduced the distraction of users when driving compared with the traditional touch-based method.Article highlightsThe user's preferred in-vehicle media control task gestures were obtained through the user elicitation method.Constructed an in-vehicle secondary task control prototype system based on gesture recognition.The eye movement experiment proved that the prototype system can reduce the user's distraction when operating the in-vehicle's secondary tasks.

Influences of Gesture-Based Mobile Phone Use While Driving

The use of mobile phones while driving has been a hot topic in the field of driving safety for decades. Although there are few studies on the influence of gesture control on in-vehicle secondary tasks, this study aims to investigate the impact of gesture-based mobile phone use without touching while driving from the perspective of multiple-resource workload owing to visual, auditory, cognitive, and psychomotor resource occupation. A novel gesture control technique was adopted for secondary task interactions, to recognize the gestures of drivers. An experiment was conducted to study the influences of two interaction modes, traditional touch-based mobile phone interaction and gesture-based mobile phone interaction, on driving behavior in three different cognitive level task groups. The results indicate that gesture-based mobile phone interaction can improve driving performance with regard to lateral position-keeping ability and steering wheel control; nevertheless, it has no significant impact on longitudinal metrics such as driving speed, driving speed variation, and throttle control variation. Gesture-based mobile phone interactions have a larger effect on secondary tasks with medium cognitive load but not on actual operation tasks. It was also verified that the performance of gesture-based mobile phone interaction was better in secondary mobile phone tasks such as switching (e.g., switching songs) and adjusting (e.g., adjusting volume) than the traditional interaction mode. This study provides the theoretical and experimental support for human–computer interaction using gesture-based mobile phone interactive control in future automobiles.

Design Guidelines for the Size and Length of Chinese Characters Displayed in the Intelligent Vehicle’s Central Console Interface

In order to ensure the driver’s safe driving, the human–computer interaction interface of an intelligent vehicle needs to convey important information. The text is an important carrier of this kind of information. The design criteria of English characters have been widely discussed, including the color, meaning, size and length. However, design guidelines for Chinese characters in central consoles of vehicles have rarely been discussed from a human–computer interaction perspective. In this paper, we investigated the size and the length of Chinese characters in the intelligent vehicle’s central control screen, based on international design guidelines and standards. The experiment involved 30 participants performing simulated in-vehicle secondary tasks. The result from the experiments shows that the usability of characters increases and the driver’s workload decreases as the characters get larger and shorter. We also propose a set of recommended values for the size and length of Chinese characters in this context. Future work will focus on providing design guidelines for other aspects of HMI design in intelligent vehicles.

Adverse Behavioral Adaptation to Adaptive Forward Collision Warning Systems: An Investigation of Primary and Secondary Task Performance

Advanced driver assistance systems can effectively support drivers but can also induce unwanted effects in behavior. The present study investigates this adverse behavioral adaptation in adaptive Forward Collision Warning (FCW) systems. Other than conventional FCW systems that provide warnings based on static Time-To-Collision (TTC) thresholds, adaptive FCW systems consider the driver’s need for support by adjusting warning thresholds according to distraction. A neglected question is how drivers adapt their behavior when they use adaptive FCW systems under realistic conditions, i.e., when warnings occur infrequently but system functionality is anticipated. Forty-eight participants drove with two different FCW systems (adaptive vs. non-adaptive) while working on a secondary in-vehicle task in a driving simulator. During the main part of the experiment, no brake events occurred and hence FCW functioning was largely anticipated. Additionally, visual system feedback about the driver’s distraction state was manipulated between groups. Participants had significantly shorter minimal time-headways and TTCs when driving with the adaptive relative to the non-adaptive system. Participants with system feedback about distraction state spent generally more time with engaging in the secondary task. These results indicate behavioral adaptation which, however, is restricted to the task that is specifically supported by the system, namely longitudinal control.

Queueing Network Based Driver Model for Varying Levels of Information Processing

With growing interest in the topic of smart computing and context-aware services, identifying driver's intention has become important in automotive industry. Among existing studies on driver assistance systems, few studies have used the concept of the levels of processing (LOP) to understand driver's information processing. This paper introduces experimental and computational studies that connect human behavior data to internal goal of a driver when an in-vehicle task involves visual search. In the case of consuming information displayed on a car instrument cluster, we considered two levels of information processing, perceptual and cognitive. Through an empirical study, we observed different human oculomotor behaviors that depend on the LOP by evaluating the reaction time (RT) and eye movement patterns. Further, we suggested different ways of information processing that can be represented as different routes in the underlying cognitive architecture of a queueing network. Simulation study demonstrated that trends in simulated oculomotor behavior were similar to those observed in the experiment, that is, RT at the cognitive LOP was shorter than that at the perceptual LOP, while eyes fixated for shorter duration and with lower frequency. In terms of application, this study reveals the possibility of using human-generated data for evaluating the innate purposes of drivers. Together with further development of sensors and invasive computing, the approach proposed in this paper could assist the realization of cognitive cars by understanding drivers’ intent using eye tracking technology.

Attention maintenance training: Are young drivers getting better or being more strategic?

Young drivers are particularly poor at maintaining attention to the forward roadway where imminent hazards may occur. Existing training programs such as FOrward Concentration and Attention Learning (FOCAL) have been shown to improve young drivers’ attention maintenance performance. The current study examines two competing hypotheses for the effectiveness of FOCAL: 1) Drivers disregard the secondary task to focus on maintaining attention, or 2) FOCAL improves drivers’ multitasking ability on the driving and the secondary tasks. FOCAL- and placebo-trained drivers navigated through four distinct scenarios in a driving simulator. During each scenario, they were asked to perform a secondary task interacting with a mock in-vehicle navigation system. Results showed that FOCAL improved driver attention maintenance performance and, surprisingly, their secondary task performance. These results suggest the possibility that FOCAL in fact increases not only their ability to maintain their attention to the forward roadway but also a drivers’ multitasking performance. Future works should use a variety of in-vehicle tasks with different visual processing demands to determine the generalizability of the current finding.

Applying a User-Centered Method to Develop 3D Gestural Inputs for In-Vehicle Tasks

Gestures are a natural input method for human communication and may be effective for drivers to interact with in-vehicle infotainment systems (IVIS). Most of the existing work on gesture-based human-computer interaction (HCI) in and outside of the vehicle focus on the distinguishability of computer systems. The purpose of this study was to identify gesture sets that are used for IVIS tasks and to compare task times across the different functions for gesturing and touchscreens. Task times for user-defined gestures were quicker than for a novel touchscreen. There were several functions that resulted in relatively intuitive gesture mappings (e.g., zooming in and zooming out on a map) and others that did not have strong mappings across participants (e.g., decreasing volume and playing the next song). The findings of this study suggest that user-centric gestures can be utilized to interact with IVIS systems instead of touchscreens, and future work should evaluate how to account for variability in intuitive gestures. Understanding the gesture variability among the end users can support the development of an in-vehicle gestural input system that is intuitive for all users.

Feel the noise: Mid-air ultrasound haptics as a novel human-vehicle interaction paradigm

Focussed ultrasound can be used to create the sensation of touch in mid-air. Combined with gestures, this can provide haptic feedback to guide users, thereby overcoming the lack of agency associated with pure gestural interfaces, and reducing the need for vision – it is therefore particularly apropos of the driving domain. In a counter-balanced 2 × 2 driving simulator study, a traditional in-vehicle touchscreen was compared with a virtual mid-air gestural interface, both with and without ultrasound haptics. Forty-eight experienced drivers (28 male, 20 female) undertook representative in-vehicle tasks – discrete target selections and continuous slider-bar manipulations – whilst driving. Results show that haptifying gestures with ultrasound was particularly effective in reducing visual demand (number of long glances and mean off-road glance time), and increasing performance (shortest interaction times, highest number of correct responses and least ‘overshoots’) associated with continuous tasks. In contrast, for discrete, target-selections, the touchscreen enabled the highest accuracy and quickest responses, particularly when combined with haptic feedback to guide interactions, although this also increased visual demand. Subjectively, the gesture interfaces invited higher ratings of arousal compared to the more familiar touch-surface technology, and participants indicated the lowest levels of workload (highest performance, lowest frustration) associated with the gesture-haptics interface. In addition, gestures were preferred by participants for continuous tasks. The study shows practical utility and clear potential for the use of haptified gestures in the automotive domain.

Evaluating secondary input devices to support an automotive touchscreen HMI: A cross-cultural simulator study conducted in the UK and China

Touchscreen Human-Machine Interfaces (HMIs) are a well-established and popular choice to provide the primary control interface between driver and vehicle, yet inherently demand some visual attention. Employing a secondary device with the touchscreen may reduce the demand but there is some debate about which device is most suitable, with current manufacturers favouring different solutions and applying these internationally. We present an empirical driving simulator study, conducted in the UK and China, in which 48 participants undertook typical in-vehicle tasks utilising either a touchscreen, rotary-controller, steering-wheel-controls or touchpad. In both the UK and China, the touchscreen was the most preferred/least demanding to use, and the touchpad least preferred/most demanding, whereas the rotary-controller was generally favoured by UK drivers and steering-wheel-controls were more popular in China. Chinese drivers were more excited by the novelty of the technology, and spent more time attending to the devices while driving, leading to an increase in off-road glance time and a corresponding detriment to vehicle control. Even so, Chinese drivers rated devices as easier-to-use while driving, and felt that they interfered less with their driving performance, compared to their UK counterparts. Results suggest that the most effective solution (to maximise performance/acceptance, while minimising visual demand) is to maintain the touchscreen as the primary control interface (e.g. for top-level tasks), and supplement this with a secondary device that is only enabled for certain actions; moreover, different devices may be employed in different cultural markets. Further work is required to explore these recommendations in greater depth (e.g. during extended or real-world testing), and to validate the findings and approach in other cultural contexts.

Voice Control Tasks on Cognitive Workload and Driving Performance: Implications of Modality, Difficulty, and Duration

The effects of an in-vehicle voice control system (VCS) on cognitive workload and driving performance were evaluated using a driving simulator study with 24 participants. Participants were asked to perform two types of in-vehicle tasks while driving: voice-command based radio and navigation tasks. The tasks were of two difficulty levels (easy, hard) with half of the tasks audio only and the other half audio with a visual display. Cognitive workload was measured using a tactile detection response task (TDRT) and a revised remote detection response task (RDRT). Driving performance was measured using standard deviation of lateral position (SDLP) and standard deviation of vehicle speed. An analysis of covariance for standard deviation of speed was used to examine the effects of VCS display with task duration as the covariate. Three separate mixed models were used to evaluate the variation in SDLP, response time, and misses given VCS tasks and detection response task (DRT) type. No significant differences were observed between TDRT and RDRT, suggesting both methods capture congruent measures of workload and that neither interfere with driving performance. Engagement with each hybrid VCS task was significantly associated with an increase in DRT misses above baseline driving. Additionally, hybrid display tasks had larger variations in vehicle speed and were completed more quickly than equivalent audio only tasks. Increasing values of task duration were associated with greater variation in lateral vehicle position. The results of this study suggest that design of VCSs should consider the modality, temporal components, and difficulty of tasks to reduce cognitive load.

Impact of information bandwidth of in-vehicle technologies on drivers' attention maintenance performance: A driving simulator study

Previous research indicates that inexperienced drivers' ability to maintain their attention on the forward roadway during driving is poorer than experienced drivers, leading to more frequent, excessively long, off-road glances that elevate the risk of crashes. However, whether their poorer attention maintenance ability depends on complexities of in-vehicle technologies has been underexplored. This study directly manipulated information bandwidth (easy or complex) of an in-vehicle monitor and asked twenty-four drivers aged 18-21 to perform a visual number judgment task with either 5 digits (easy) or 11 digits (complex), during simulated driving. Participants had to verbally respond within 15 s whether each string of presented digits contained more odd or even digits. Eye movements were recorded using an eye tracker. Results show that the drivers produced a greater number of off-road glances and longer summed excess glance durations under a 1.5-s threshold when the in-vehicle task imposed greater information processing demand. In practice, designers of in-vehicle technologies should consider information-processing demands of in-vehicle tasks required by the technologies to minimize the frequency of excessively long off-road glances during driving.

Good intentions: drivers\u2019 decisions to engage with technology on the road and in a driving simulator

Distraction-related accidents are, more often than not, due to the drivers’ voluntarily engagement with a secondary task. Therefore, the strategic management of in-vehicle tasks and the drivers’ decision to engage with them is an important aspect of the driver distraction phenomenon that needs to be addressed. While the consequences of distracting tasks are often assessed in settings where the risk of engaging is reduced (i.e., simulators), the drivers’ decision to engage with secondary tasks is often ignored. This study assessed the drivers’ decision to engage with secondary tasks using verbal protocols to provide insights into the drivers’ intention within a naturalistic driving setting, on the road, and in a simulated driving environment. This enabled an understanding of when drivers engage with technological distractions, why they choose to do so, as well as how they may go about doing it. Different road types were found to differentially impact the drivers’ intention to engage, as did the types of secondary tasks, with some tasks having an increased willingness to engage compared to others. Factors that increase and/or reduce the likelihood of engaging are presented. The decisions that drivers made to engage with secondary tasks in the simulator were found to correlate strongly to their decision to so on the road. This provides support for the use of simulators when assessing the drivers’ decision to engage with secondary tasks. The effect of verbal protocols on the drivers’ speed metrics was assessed to determine how they may have affected their driving performance.

User experience comparison among touchless, haptic and voice Head-Up Displays interfaces in automobiles

This paper evaluates driving experiences when using a Head-Up Display (HUD) system by different interaction methods, since HUDs in automotive market do not follow standard interaction methods (Betancur et al. in Int J Interact Des Manuf 12(1):199–214, [3]), and therefore it is difficult to identify what types of these methods are the most appropriated in terms of usability for being implemented in HUDs. This paper focuses on comparing the mental workload that a driver could experience while interacting with a specific HUD visual interface by touchless gestures, haptic or voice methods. Then, test subjects (n = 15) performed conjointly a driving activity and a set of in-vehicle tasks by using these interaction methods throughout a HUD. The experiments show that the haptic method was better accepted than the touchless gestures and voice methods. Nevertheless, the touchless gestures method was explored under some specific usability configurations, in order to demonstrate that it was not significantly different from the haptic method.

The effect of information availability in a user interface (UI) on in-vehicle task performance: A pilot study

When performing an in-vehicle operation, the information display is a crucial element that decides the operator's control behaviors. The interaction between information presentation and operator's in-vehicle task performance is critical for vehicle safety. The question arises: What is the optimum availability of information? The aim of this study is to investigate the relationships between different information availabilities and operation performances as operators interact with an in-vehicle user interface (UI) to complete a simulated train driving task. The results show that some types of information significantly affect the operator's task performance. Information that improved the task performances were closely associated with specific task requirements. The participants' task performance indicated that they had a better awareness of a task situation and were better at controlling their virtual vehicle when completing time and process-restricted tasks with limited, relevant information. These results support the conclusion that more information does not always promote accuracy and safety during the task completion. Relevance to industryThis study presents a unique insight into the information display of in-vehicle UI design for vehicle control. The information available within an in-vehicle interface should be optimized by performance assessments to provide more useful information; this has significant positive effect on operator performance from an accuracy and safety perspective. Thus, this study provides advice for designing an in-vehicle UI.

Impact of training and in-vehicle task performance on manual control recovery in an automated car

Automated driving (AD) introduces new skills needs for drivers to handle manual control recovery (MCR). In the scope of traffic safety, such skills should be investigated before this technology is available on public roads, especially in critical situations such as emergencies. The aim of the present study is to examine to what extend lack of training and in-vehicle task performance may impair MCR in a fully automated car. This study explores AD in a driving simulator, focusing on emergency MCR performance and trust in this system. 113 participants were involved. Training (simple vs. elaborated) and in-vehicle task performance (with vs. without) were manipulated. The MCR method (hands and feet used simultaneously or not) and its impact on response time and the number of pedals interactions were also examined. Training improved response time and decreased the number of interactions with the pedals. Moreover, the elaborated training group had fewer interactions with the pedals in the second and last system failure than in the first one. Performing an in-vehicle task slowed response time but did not significantly affect the number of interactions with the pedals. Response time was faster when hands and feet were used simultaneously in MCR. Elaborated training influenced trust in AD. Training drivers, using practice of the system and explaining its underlying logic, contributed to improve the human-automation performance. These results suggest the benefits of a training policy to enhance MCR performance and optimize trust in the system.