In-vehicle Infotainment Research Articles

L-GraphSAGE: A Graph Neural Network-Based Approach for IoV Application Encrypted Traffic Identification

Recently, with a crucial role in developing smart transportation systems, the Internet of Vehicles (IoV), with all kinds of in-vehicle devices, has undergone significant advancement for autonomous driving, in-vehicle infotainment, etc. With the development of these IoV devices, the complexity and volume of in-vehicle data flows within information communication have increased dramatically. To adapt these changes to secure and smart transportation, encrypted communication realization, real-time decision-making, traffic management enhancement, and overall transportation efficiency improvement are essential. However, the security of a traffic system under encrypted communication is still inadequate, as attackers can identify in-vehicle devices through fingerprinting attacks, causing potential privacy breaches. Nevertheless, existing IoV traffic application models for encrypted traffic identification are weak and often exhibit poor generalization in some dynamic scenarios, where route switching and TCP congestion occur frequently. In this paper, we propose LineGraph-GraphSAGE (L-GraphSAGE), a graph neural network (GNN) model designed to improve the generalization ability of the IoV application of traffic identification in these dynamic scenarios. L-GraphSAGE utilizes node features, including text attributes, node context information, and node degree, to learn hyperparameters that can be transferred to unknown nodes. Our model demonstrates promising results in both UNSW Sydney public datasets and real-world environments. In public IoV datasets, we achieve an accuracy of 94.23%(↑0.23%). Furthermore, our model achieves an F1 change rate of 0.20%(↑96.92%) in α train, β infer, and 0.60%(↑75.00%) in β train, α infer when evaluated on a dataset consisting of five classes of data collected from real-world environments. These results highlight the effectiveness of our proposed approach in enhancing IoV application identification in dynamic network scenarios.

How Does Aesthetic Design Affect Continuance Intention in In-Vehicle Infotainment Systems? An Exploratory Study

With the digitalization of automobile cabins, in-vehicle infotainment systems play an increasingly pivotal role in vehicles. This study aims to understand how the interface aesthetics of the in-vehicle infotainment system affect users’ continuance intention in in-vehicle infotainment systems. Based on a literature review, a theoretical model was constructed, encompassing seven latent variables: simplicity, colorfulness, diversity, craftsmanship, functional value, emotional value, and continuance intention. Analysis of 243 valid questionnaires revealed that both functional and emotional values serve as crucial mediators in encouraging users to continue using the in-vehicle infotainment system, with functional quality being the core element. Moreover, simplicity, diversity, and craftsmanship indirectly affect continuance intention by influencing both functional and emotional values. These findings underscore the need for future designers of in-vehicle infotainment systems to focus on interface functionality and emotional needs, and to ensure that content is concise, engaging, user-friendly, and visually polished, thereby enhancing the user experience and fostering the intention to continue usage.

A gaze-based driver distraction countermeasure: Comparing effects of multimodal alerts on driver's behavior and visual attention

This study, introduces and evaluates different countermeasures using real-time eye-tracking data. The countermeasures detect when driver gaze deviates from the road for longer than a predetermined threshold and then redirect the driver's attention back to the road. The countermeasures include bimodal and trimodal alerts using combinations of auditory, tactile, and visual modalities. These countermeasures showcase the utility of adopting eye-tracking technologies in the context of driver monitoring and advanced driver's assistance systems. They enhance safety as a safeguard for the increased use of devices such as in-vehicle infotainment systems. Results show that countermeasures effectively redirect drivers’ attention to the road, with higher on-road gaze time. Additionally, bimodal alerts that include the visual modality are less effective at redirecting participants’ gaze on-road and result in poorer driving performance.

Head-up Displays Improve Drivers’ Performance and Subjective Perceptions with the In-Vehicle Gesture Interaction System

In-vehicle infotainment systems can cause various distractions, increasing the risk of car accidents. To address this problem, mid-air gesture systems have been introduced. This study investigated the potential of a novel interface that integrates a Head-Up Display (HUD) with auditory displays (spearcons: compressed speech) in a gesture-based menu navigation system to minimize visual distraction and improve driving and secondary task performance. The experiment involved 24 participants who navigated through 12 menu items using mid-air gestures while driving on a simulated road under four conditions: HUD (with, without spearcons) and Head-Down Display (HDD) (with, without spearcons). Results showed that the HUD condition significantly outperformed the HDD condition in participants’ level 1 situation awareness, perceived workload, menu navigation performance, and system usability. However, there were trade-offs on visual fixation duration on the menu, and lane deviation. These findings will guide future research in developing safer and more effective HUD-supported in-vehicle gesture interaction systems.

Research on the Antecedents of Data Privacy Concern Toward Intelligent Connected Vehicles

Depending on Internet of Vehicle (IoV) and on-board units in 5 G, intelligent connected vehicles (ICVs) gradually become the digital infrastructure and data hub to provide personalized in-vehicle infotainment (IVI), which naturally collect, use, and share the data related to users’ bio-informatics, travel patterns, and usage habits. The privacy concern (PC) will be generated when using those data services. In this article, we construct the structural equation model (SEM) to study the influencing factors of users’ PC and self-disclosure behavior (SD) regarding to ICV data services from the personality traits (PTs) perspective. A total of 481 valid questionnaires from 700 IVI users was collected. Results show that extraversion (EXT), conscientiousness (CNS), neuroticism (NEUR), privacy invasion experience (PIE), and privacy self-efficacy (PE) have significant impacts on PC. Besides, PC does not directly affect SD, but is indirectly affected by limiting profile visibility (LPV). Through revealing the relationship among PTs, PC and SD toward ICV data services, we aim to provide insights for vehicle data governance to facilitate the human-vehicle interaction.

STRIDE-Based Cybersecurity Threat Modeling, Risk Assessment and Treatment of an In-Vehicle Infotainment System

In modern automobiles, the infotainment system is crucial for enhancing driver and passenger capabilities, offering advanced features such as music, navigation, communication, and entertainment. Leveraging Wi-Fi, cellular networks, NFC, and Bluetooth, the system ensures continuous internet connectivity, providing seamless access to information. However, the increasing complexity of IT connectivity in vehicles raises significant cybersecurity concerns, including potential data breaches and exposure of sensitive information. To enhance security in infotainment systems, this study applied component-level threat modeling to a proposed infotainment system using the Microsoft STRIDE model. This approach illustrates potential component-level security issues impacting privacy and security concerns. The study also assessed these impacts using SAHARA and DREAD risk assessment methodologies. The threat modeling process identified 34 potential security threats, each accompanied by detailed information. Moreover, a comparative analysis is performed to compute risk values for prioritizing treatment, followed by recommending mitigation strategies for each identified threat. These identified threats and associated risks require careful consideration to prevent potential cyberattacks before deploying the infotainment system in automotive vehicles.

Multi-Sensor-Assisted Low-Cost Indoor Non-Visual Semantic Map Construction and Localization for Modern Vehicles.

With the transformation and development of the automotive industry, low-cost and seamless indoor and outdoor positioning has become a research hotspot for modern vehicles equipped with in-vehicle infotainment systems, Internet of Vehicles, or other intelligent systems (such as Telematics Box, Autopilot, etc.). This paper analyzes modern vehicles in different configurations and proposes a low-cost, versatile indoor non-visual semantic mapping and localization solution based on low-cost sensors. Firstly, the sliding window-based semantic landmark detection method is designed to identify non-visual semantic landmarks (e.g., entrance/exit, ramp entrance/exit, road node). Then, we construct an indoor non-visual semantic map that includes the vehicle trajectory waypoints, non-visual semantic landmarks, and Wi-Fi fingerprints of RSS features. Furthermore, to estimate the position of modern vehicles in the constructed semantic maps, we proposed a graph-optimized localization method based on landmark matching that exploits the correlation between non-visual semantic landmarks. Finally, field experiments are conducted in two shopping mall scenes with different underground parking layouts to verify the proposed non-visual semantic mapping and localization method. The results show that the proposed method achieves a high accuracy of 98.1% in non-visual semantic landmark detection and a low localization error of 1.31 m.

Safety Aspects of In-Vehicle Infotainment Systems: A Systematic Literature Review from 2012 to 2023

This systematic literature review investigates the safety aspects of in-vehicle infotainment systems (IVISs) from 2012 to 2023, analyzing 96 studies. IVISs have significantly evolved, incorporating technologies such as navigation systems, parking assistance, and video games. However, these innovations introduce safety concerns like driver distraction and cognitive overload. This review identifies six primary safety issues: driving distraction, situational awareness, cognitive load, driving performance, interaction success, and emotional state. Head-down displays and touchscreens often have negative safety implications, while speech-based interfaces and Bluetooth-integrated systems are generally considered safer. Suggested improvements include enhancing interface design for touchscreens and exploring gesture-based alternatives. Despite these developments, significant gaps remain in real-world evaluations and studies in diverse driving conditions, highlighting the need for standardized manufacturing norms. Addressing these issues is essential for creating future IVIS that are both reliable and safe. This review serves as a foundation for future research, safety regulations, and design principles aimed at improving IVIS safety. Overcoming these challenges will require a multi-faceted approach that considers user-friendly design, adaptive technologies, and predictive analytics. The goal is to balance technological advancements with road safety, ensuring that IVISs contribute to a safer driving experience without compromising convenience and functionality.

User experience heuristics for in-vehicle infotainment systems

The use of digital and interactive panels incorporated into various transportation vehicles is a trend that is attracting increasing interest from users. These panels called “in-vehicle infotainment systems” (IVIS) offer a variety of functionalities, from listening to music to visualizing a travel route. However, IVISs must be designed with special care, as they should not distract users from their main objective: driving the vehicle. Given the above, it is essential to design a user-friendly and intuitive IVIS that does not force the user to think about how to use it or create distractions.To facilitate the detection of specific IVIS-related problems and thus improve the user experience (UX), we proposed a new set of 12 heuristics named “UXH-IVIS: user experience heuristics for in-vehicle infotainment systems”. This set is composed of 12 heuristics and was developed using an 8-stage methodology. To create UXH-IVIS, we considered the specific features that describe IVIS, UX factors, and existing sets of heuristics. The set has been validated through a heuristic evaluation in one iteration. In general, the results obtained in the validation corroborate the usefulness of the set, since UXH-IVIS allows to detect severe problems related to this type of systems. As future work, we intend to perform further validations with experts and heuristic evaluations to verify that the proposed set is effective, efficient, and allows the evaluation of different types of IVIS.

Sonically-enhanced in-vehicle air gesture interactions: evaluation of different spearcon compression rates

Driver distraction is a major contributor to road vehicle crashes, and visual distraction caused by using in-vehicle infotainment systems (IVIS) degrades driving performance and increases crash risk. Air gesture interfaces have been developed to mitigate driver distraction, and using auditory displays showed a decrease in off-road glances and an improved perceived workload. However, the potential of auditory display was not fully investigated. The present paper presents directional research in the design of auditory displays for air-gesture IVIS through a dual-task experiment of driving a simulator and air-gesture menu navigation. Twenty-four participants utilized the in-vehicle air gesture interfaces while driving with four auditory display conditions (text-to-speech, 70% compressed spearcons, 40% compressed spearcson, and no sound). The results showed that 70% spearcon reduced visual distraction and increased menu navigation accuracy but with increased navigation time. 70% spearcon was most preferred by the participants. Driving performance or workload did not show any difference among the conditions. Implications are discussed with the design guidelines for future implementations.

Machine Learning-assisted Distance Based Residual Energy Aware Clustering Algorithm for Energy Efficient Information Dissemination in Urban VANETs

A Vehicular Ad-hoc Network (VANET) is an essential component of intelligent transportation systems in the building of smart cities. A VANET is a self-configure high mobile and dynamic potential wireless ad-hoc network that joins all vehicle nodes in a smart city to provide in-vehicle infotainment services to city administrators and residents. In the smart city, the On-board Unit (OBU) of each vehicle has multiple onboard sensors that are used for data collection from the surrounding environment. One of the main issues in VANET is energy efficiency and balance because the small onboard sensors can’t be quickly recharged once installed on On-board Units (OBUs). Moreover, conserving energy stands out as a crucial challenge in VANET which is primarily contingent on the selection of Cluster Heads (CH) and the adopted packet routing strategy. To address this issue, this paper proposes distance and energy-aware clustering algorithms named SOMNNDP, which use a Self-Organizing Map Neural Network (SOMNN) machine learning technique to perform faster multi-hop data dissemination. Individual Euclidean distances and residual node energy are considered as mobility parameters throughout the cluster routing process to improve and balance the energy consumption among the participating vehicle nodes. This maximizes the lifetime of VANET by ensuring that all intermediate vehicle nodes use energy at approximately the same rate. Simulation findings demonstrate that SOMNNDP improves Quality of Service (QoS) better and consumes 17% and 14% less energy during cluster routing than distance and energy-aware variation of K-Means (KM) and Fuzzy C-Means (FCM) called KMDP and FCMDP respectively.

RFDrive: Tagged Human-Vehicle Interaction for All

Human–vehicle interaction is an important factor for safe driving. The driver needs to interact with the in-vehicle steering wheel and infotainment system properly during driving. Specifically, driving guidelines require the driver to hold the steering wheel at the 3 o’clock and 9 o’clock positions. Moreover, the in-vehicle infotainment system should be more adaptive for the driver and front-seat passenger during driving (i.e., the in-vehicle infotainment system should be part and even fully disabled for the driver, whereas the front-seat passenger should be able to enjoy the full in-vehicle infotainment system). However, affordable vehicles are usually designed to achieve basic driving functions without considering safe human–vehicle interactions, which require an add-on, affordable, and ready-to-use human–vehicle interaction monitoring system. In this article, we present RFDrive, a system that can simultaneously locate the driver’s hand positions on the steering wheel and automate in-vehicle infotainment system touch discrimination for safe driving using commodity passive RFID tags. Since these commodity passive RFID tags are low cost (i.e., around 5 cents per tag), battery free, and are small, like a sticker, our design will enable not only safe driving but is also low cost, which can lead to sustainable solutions. To do so, we attach RFID tags on the steering wheel for the driver’s hand position location and attach RFID tags on the roof of the vehicle’s interior for in-vehicle infotainment system touch discrimination (i.e., differentiating the driver’s infotainment system touch and front-seat passenger’s infotainment system touch). However, the wheel steering will distort the wireless channel-based driver’s hand position location on the steering wheel. Thus, we propose a novel tag ID-based algorithm to locate the driver’s hand position on the steering wheel by harnessing the human body as part of the RFID tag’s antenna. Since the in-vehicle infotainment system touch from the driver or front-seat passenger will affect different RFID tags attached to the roof of the vehicle’s interior, we propose to use the differential amplitude of backscattered signals from all the tags to discriminate in-vehicle infotainment system touch sources. Our experiments show that RFDrive can achieve the average accuracy of 0.98 and 0.98 for in-vehicle touch source discrimination and driver’s hand position location, respectively.

A vehicle firmware security vulnerability: an IVI exploitation

In the last years, the increasing vehicular technology has led to a surge in cybersecurity attacks, particularly regarding connected vehicles and their vulnerable infotainment systems. This paper explores the vulnerabilities within an In-Vehicle Infotainment (IVI) system firmware, focusing on the Gen5W_L multimedia device utilized in Hyundai, Kia, and Genesis vehicles. Leveraging reverse engineering techniques, the study uncovers several security issues within the firmware, allowing for the creation and installation of custom firmware. The paper introduces Chimaera, a reverse engineering attack targeting the IVI system firmware, which exploits a printf vulnerability introduced by statically compiling the mbedtls library. After a reverse engineering analysis and injecting malicious assembly code directly in the IVI firmware, the attack enables the extraction of sensitive information from the firmware files like the cryptographic keys to install a customized firmware. The research shows that exploiting these vulnerabilities facilitates the installation of insecure firmware, granting attackers unauthorized access to vehicle functionalities. This includes the possibility to inject CAN bus messages, potentially compromising critical vehicle systems such as radio controls and safety features. The findings underscore the need for robust cybersecurity measures within automotive systems and highlight the significance of vulnerability assessment to mitigate security risks in connected vehicles.

Driving Distraction Evaluation Model of In-Vehicle Infotainment Systems Based on Driving Performance and Visual Characteristics

With the rapid development of automotive electronic technology, increasingly car enterprises choose to carry touchscreen in-vehicle infotainment systems (IVISs) in smart cars. The integration of navigation, music, telephone, and other functions into touchscreen IVISs has greatly enriched the driving experience. However, the design of touchscreen IVISs, which lacks consideration of driver human–computer interaction leads to increased driving distraction. The current discussion is limited to evaluating the overall distractions of the secondary task, and there is a lack of research on classifying and summarizing the types of driving distractions. Therefore, this paper proposes a new method to evaluate the degree of driving distraction during the interaction with touchscreen IVISs based on driving performance and visual characteristics. A driving simulation environment was built to conduct psychological experiments on three types of driving distraction, namely visual, manual, and cognitive distraction, respectively, to obtain the impact of different types of driving distraction by using touchscreen IVISs. Combining the criteria importance through intercriteria correlation method with fuzzy comprehensive evaluation to establish a driving distraction evaluation method, we quantify the driving distraction caused by the touch screen IVIS. The results of this study indicate that the proposed driving distraction evaluation model by using driving performance and visual characteristics can effectively evaluate the degree of driving distraction caused by the touchscreen IVIS and provide reference and specification for IVIS safety design from a human–computer interaction perspective.

Fine-Grained Satisfaction Analysis of In-Vehicle Infotainment Systems Using Improved Kano Model and Cumulative Prospect Theory

Fine-Grained Satisfaction Analysis of In-Vehicle Infotainment Systems Using Improved Kano Model and Cumulative Prospect Theory

Driving Adoption of Advanced IC Packaging in Automotive Applications

For years automotive semiconductors, have been governed by small, low power Integrated Circuits (ICs) purpose built for specific functions. These Microcontroller Units (MCUs), deployed on the edge of the compute topology are typically low-power, low-performance parts. Built on legacy nodes (130nm+) on well-established packaging technology (often wirebond die on QFN-style packages), they demonstrated incremental improvements from a silicon and packaging perspective. With the adoption of software defined vehicles (SDV), the evolution of advanced driver assist systems (ADAS), and feature rich in-vehicle infotainment (IVI), the per vehicle compute power has grown exponentially per vehicle, colloquially, known as “data centers on wheels”. With this growth has come the need to adopt newer packaging techniques to improve system latency, bandwidth, compute power, and IP integration and reducing system footprint via 3D stacking – all fabricated on leading edge nodes. Advanced packaging has seen tremendous growth and adoption in the data center and client ecosystems – the next hurdle is addressing the safety and reliability focused requirements of automotives; extended useful lifespan of 10yrs+, and weathering harsher environmental conditions of thermal extremes, vibration, and EMC, and mixed IP integration. This talk will address the areas of focus for adopting heterogenous integration and advanced automotive packaging for automotives such as, qualification and formulation of materials to minimize traditional packaging yield concerns, introduction of new thermal interface materials aligned with new liquid cooled ECUs and testing to ensure zero defects across chiplets and memory; with broader test coverage and the adoption of Known Good Die (KGD). Additionally, the talk will cover the importance of driving industry standardization regarding die-to-die communication protocols, form factors, power delivery, system integration, and qualifying/enveloping new failure modes from the current baseline AEC-Q100 requirements brought on by various 2.5D and 3D packaging.

SP-E: Security Evaluation Framework of In-vehicle Infotainment System based on Threat Analyses and Penetration Tests

With the increasing requirement of people, the functions of in-vehicle infotainment systems are becoming more and more abundant, and their security also affects the safety of vehicles. Therefore, it is more and more important to evaluate the security of the IVI system. This paper proposes a security evaluation framework for in-vehicle infotainment systems based on threat analyses and penetration tests. By constructing the data flow diagram of application scenarios, analyzing threats, combing the attack link diagram, combining white-box audit and black-box test, we use the characteristics of high efficiency of automatic tools and high accuracy of manual methods to set factor sets, and then the whole IVI system is evaluated by Analytic Hierarchy Process and Fuzzy Comprehensive Evaluation.

Perspectives on Big Data, Cloud-Based Data Analysis and Machine Learning Systems

Huge amounts of digital data are continuously generated and collected from different sources, such as sensors, cameras, in-vehicle infotainment, smart meters, mobile devices, social media platforms, and web applications and services [...]

Empowering Driver-Passenger Collaboration: Designing In-Car Systems with a Focus on Social Connectedness, Fairness, and Team Performance

Driving a car can be difficult when it comes to distractions caused by operating the in-vehicle infotainment system (IVIS). In-car passengers often help with performing IVIS-related tasks. However, an IVIS is often not designed with a focus on task collaboration. In this article, we focus on how to design in-car systems with the goal to support collaboration between a driver and a front-seat passenger. Based on infotainment-oriented tasks, we initially explore five key collaborative control concepts by means of an IVIS which differ from each other in terms of the number of available IVIS screens (one or two), access to menus (restricted and unrestricted), and the nature of performing tasks in parallel or one after the other. Results from a simulator study with N = 16 pairs show significant effects of the concepts on social collaboration in terms of perceived social connectedness (measured with sub-dimensions connectedness, affiliation, belongingness, companionship), team performance (coordination effectiveness and team cohesion), and fairness. We found that especially a dedicated passenger IVIS screen empowers front-seat passengers, reduces power dynamics, supports fairness, and minimizes driver distraction (caused by interacting passengers). We discuss the implications of these findings and posit recommendations to design future IVIS in passenger cars with improved driver-passenger collaboration by explicitly designing for balanced power roles, situational awareness, active communication, and a balance between drivers’ privacy and trust toward the passenger. Additionally, we outline a systematic overview of future work to explore the research field of driver-passenger collaboration in more breadth and depth.

Deep Learning-Based Business Recommendation System in Intelligent Vehicles

The advancements in intelligent vehicle technologies are facilitating the growth of information technology (IT) platforms, unlike conventional automobiles. In-vehicle-infotainment (IVI) is becoming an appealing element in intelligent vehicles as it offers various experiences to users; however, it requires personalized services to provide even more sophisticated user experiences. It is supposed that passengers search for businesses that provide products or services they found interesting in videos played via IVIs while the vehicle is driving autonomously. In that case, it could be more effective to use images that can express the user’s preference as a query for the search than to utilize texts such as product names. Accordingly, this study proposes a recommendation system that informs users of businesses near an intelligent vehicle when a passenger inputs an image of a product or service into an IVI system. The proposed recommendation system involves training deep learning-based image classification models with the user’s interest images to classify the category, measure the similarity with the business category using Word2vec, and finally provide the locations of the businesses with a high degree of similarity via IVI, using a smartphone. The experimental results indicated that the user’s interest image exhibited 85% accuracy for category classification via the EfficientNet B0 model, while the similarity between the image and business categories using Word2vec was particularly high in the business category similar to the actual image category.