Car Environment Research Articles

Azimuth perception of virtual sources in automotive environment : speech and musical stimulus

The spatial reproduction of musical media in car environment presents a number of challenges, due to the fact that the loudspeakers are not symmetrical in relation to the listeners and to the unique acoustic characteristics. In particular, the positioning of virtual sources is difficult to apprehend, unlike in conventional stereophonic configurations. In an initial study, it was shown that an azimuth perception model, the extended energy vector, produced results that were consistent with experimental data. As the first experiment was carried out using only a short burst of pink noise, it seemed interesting to compare this model with other data from more natural signals. New experiments were therefore carried out using audio samples such as speech and instruments. The subjects, seated in the driving seat, pointed their noses in the perceived direction of the signal, their head movements being recorded with a headtracker. The model will be compared with these new experimental data.

Efficient Management of Containers for Software Defined Vehicles

Containerization technology, such as Docker, is gaining in popularity in newly established software-defined vehicle architectures (SDVA). However, executing those containers can quickly become computationally expensive in constrained environments, given the limited CPU, memory, and energy resources in the Electric Control Units (ECU) of SDVA. Consequently, the efficient management of these containers is crucial for enabling the on-demand usage of the applications in the vehicle based on the available resources while considering several constraints and priorities, including failure tolerance, security, safety, and comfort. In this paper, we propose a dynamic software container management approach for constrained environments such as embedded devices/ECUs in SDVA within smart cars. To address the conflicting objectives and constraints within the vehicle, we design a novel search-based approach based on multi-objective optimization. This approach facilitates the allocation, movement, or suspension of containers between ECUs in the cluster. Collaborating with our industry partner, Ford Motor Company, we evaluate our approach using different real-world software-defined scenarios. These scenarios involve using heterogeneous clusters of ECU devices in vehicles based on real-world software containers and use-case studies from the automotive industry. The experimental results demonstrate that our scheduler outperforms existing scheduling algorithms, including the default Docker scheduler -Spread- commonly used in automotive applications. Our proposed scheduler exhibits superior performance in terms of energy and resource cost efficiency. Specifically, it achieves a 35% reduction in energy consumption in power-saving mode compared to the scheduler employed by Ford Motor Company. Additionally, our scheduler effectively distributes workload among the ECUs in the cluster, minimizing resource usage, and dynamically adjusts to the real-time requirements and constraints of the car environment. This work will serve as a fundamental building block in the automotive industry to efficiently manage software containers in smart vehicles considering constraints and priorities in the real world.

SLEEPINESS OR DROWZINESS AND FATIGUE DETECTION SYSTEM OF DRIVER IN THE ROAD NETWORK

On the basis of data, it is analyzed that drivers feeling drowsy are responsible for accidents on the roads.[3], posing a serious threat to public safety. To tackle this issue and take advantage of the mobile devices, a unique solution for Driver Drowsiness or sleepiness and fatigue Detection[2] system has been proposed. This solution utilizes Convolutional Neural Networks along with Flask for image/live input. By following this two-step technique, the mobile devices in the car record and evaluate the driver's current state while also ensuring their privacy. Then, boundaries verify sleepiness by comparing the data from the mobile client with the real-time input that was observed. The suggested framework uses a data fusion technique and is based on the distributed boundaries architecture, which ensures effective administration of the area of interest. This method uses flask manual input or real-time input of the car environment and the CNN model to identify driver fatigue locally based on facial expressions. With an impressive average accuracy, the framework's sleepiness detection performance is remarkable.

Evolving interpretable decision trees for reinforcement learning

In recent years, reinforcement learning (RL) techniques have achieved great success in many different applications. However, their heavy reliance on complex deep neural networks makes most RL models uninterpretable, limiting their application in domains where trust and security are important. To address this challenge, we propose MENS-DT-RL, an algorithm capable of constructing interpretable models for RL via the evolution of decision tree (DT) models. MENS-DT-RL uses a multi-method ensemble algorithm to evolve univariate DTs, guiding the process with a fitness metric that prioritizes interpretability and consistent high performance. Three different initializations for the MENS-DT-RL are proposed, including the use of Imitation Learning (IL) techniques, and a novel pruning approach that reduces solution size without compromising performance. To evaluate the proposed approach, we compare it with other models from the literature on three benchmark tasks from the OpenAI Gym library, as well as on a fertilization problem inspired by real-world crop management. To the best of our knowledge, the proposed scheme is the first to solve the Lunar Lander benchmark with both interpretability and a high confidence rate (90% of episodes are successful), as well as the first to solve the Mountain Car environment with a tree of only 7 nodes. On the real-world task, the proposed MENS-DT-RL is able to produce solutions with the same quality as deep RL policies, with the added bonus of interpretability. We also analyze the best solutions found by the algorithm and show that they are not only interpretable but also diverse in their behavior, empowering the end user with the choice of which model to apply. Overall, the findings show that the proposed approach is capable of producing high-quality transparent models for RL, achieving interpretability without losing performance.

Simulation model of a vehicle in the Simulink environment for determining fuel consumption in driving cycles

BACKGROUND: The paper shows that the problem of determining objective indicators of fuel consumption in operation based on calculation methods remains relevant. One of these methods is simulation modeling. Current state of computer technology significantly simplifies building of system simulation models, for example, using the Simulink software by Mathworks. In the process of model development, it is necessary to keep the balance of generalization and detailing, since excessive detailing leads to an unreasonably deep consideration of individual elements of the system and to a significant increase in simulation time.
 AIMS: Search for a reasonable structure of model of the driver car environment system, which makes it possible to simulate the car in driving cycles or other driving scenarios.
 METHODS: The MATLAB/Simulink simulation environment was used in the study. Based on the analysis of the car design and the capabilities of individual Simulink tools, a block diagram to study fuel consumption using the example of a passenger vehicle was synthesized. The obtained model was debugged for the stable motion simulation of the vehicle in various driving cycles using graphical analysis.
 RESULTS: The obtained results show that the developed system simulation model is well behaved. Control systems ensure the vehicle motion according to the mixed driving cycle modes in compliance with the required tolerances. It is possible to register fuel consumption and other simulation results of interest.
 CONCLUSIONS: The model showed the relative simplicity of setting up and the ability of quick update for other vehicles. A minimal amount of vehicle data is required to bring the model to an operable condition. Particular unknown parameters are supposed to be refined using identification methods.

In-car speech enhancement based on source separation technique

This work addresses the solution of localizing and enhancing hands-free speech inside the car environment. Cars have different types of sounds from outside, co-passengers dialogue and noise. To provide better-quality speech, a microphone array-based beamforming technique is used. This research work proposes the method for selected source localization, source separation, and enhancement. An estimation of the direction of arrival (DOA) to localize the signal direction and preferred direction is selected for speech enhancement. The spiral and sine-cosine algorithm (SSCA) algorithm is combined with an adaptive least mean square to adapt the system for different environments. The algorithm is implemented in hardware and tested in a real-time car environment. The results showed significant improvement in signal-to-noise ratio (SNR) of 5.2 dB and perceptual evaluation of speech quality (PESQ) of 2.3. Finally, the model is fine-tuned for the car to get better quality. The proposed technique is efficient, and results are compared with existing methods.

Low-Complexity and Accurate Noise Suppression Based on an a Priori SNR Model for Robust Speech Recognition on Embedded Systems and Its Evaluation in a Car Environment

Low-Complexity and Accurate Noise Suppression Based on an a Priori SNR Model for Robust Speech Recognition on Embedded Systems and Its Evaluation in a Car Environment

A Systematic Literature Review for Measure, Estimation, and Mitigation of Motion Sickness in Vehicle Environment

There are various studies to investigate the factors that affect driving experiences in the automobile environment and auditory experiences as variables. These factors might eventually affect users' sensitivity and performance of driving-related and non-driving-related tasks. Research on mechanical and electrical technology development is significant and has been conducted a lot. For driver- or passenger-centered auditory experiences, however, it is very important to understand people's perceptions and evaluations according to the technical design of the car sound. Therefore, in this paper, we would like to identify trends in 10 years of research on in-vehicle auditory experiences studied since 2013 and propose a human-centered automotive sound evaluation framework. The driving context, sound, and people were derived as factors affecting the car hearing experience, and specific sub-factors were derived. The results of this study are expected to provide insight into directionality in future automotive auditory.

Predicted Mean Vote of Subway Car Environment Based on Machine Learning

The thermal comfort of passengers in the carriage cannot be ignored. Thus, this research aims to establish a prediction model for the thermal comfort of the internal environment of a subway car and find the optimal input combination in establishing the prediction model of the predicted mean vote (PMV) index. Data-driven modeling utilizes data from experiments and questionnaires conducted in Nanjing Metro. Support vector machine (SVM), decision tree (DT), random forest (RF), and logistic regression (LR) were used to build four models. This research aims to select the most appropriate input variables for the predictive model. All possible combinations of 11 input variables were used to determine the most accurate model, with variable selection for each model comprising 102 350 iterations. In the PMV prediction, the RF model was the best when using the correlation coefficients square (R2) as the evaluation indicator (R2: 0.7680, mean squared error (MSE): 0.2868). The variables include clothing temperature (CT), convective heat transfer coefficient between the surface of the human body and the environment (CHTC), black bulb temperature (BBT), and thermal resistance of clothes (TROC). The RF model with MSE as the evaluation index also had the highest accuracy (R2: 0.7676, MSE: 0.2836). The variables include clothing surface area coefficient (CSAC), CT, BBT, and air velocity (AV). The results show that the RF model can efficiently predict the PMV of the subway car environment.

Estimation of Driver’s Gaze Region From Head Position and Orientation Using Probabilistic Confidence Regions

Visual attention is one of the most important aspects related to driver distraction. Estimating the driver’s visual attention can help a vehicle understand the awareness state of the driver, providing important contextual information. While estimating the exact gaze direction is difficult in the car environment, a coarse estimation of the visual attention can be obtained by tracking the head pose. Since the relation between head pose and gaze direction is not one-to-one, this paper proposes a formulation based on probabilistic models to create salient regions describing the driver’s visual attention. The area of the estimated region is small when the model has high confidence, which is directly learned from the data. We use <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Gaussian process regression</i> (GPR) to implement the framework, comparing the performance with different regression formulations such as linear regression and neural network based methods. We evaluate these frameworks by studying the tradeoff between spatial resolution and accuracy of the probability map using naturalistic recordings collected with the UTDrive platform. We observe that the GPR method produces the best result creating accurate estimations with localized salient regions. For example, the 95% confidence region is defined by an area covering 3.77% region of a sphere surrounding the driver.

Enhancing Visual Domain Randomization with Real Images for Sim-to-Real Transfer

In order to train reinforcement learning algorithms, a significant amount of experience is required, so it is common practice to train them in simulation, even when they are intended to be applied in the real world. To improve robustness, camerabased agents can be trained using visual domain randomization, which involves changing the visual characteristics of the simulator between training episodes in order to improve their resilience to visual changes in their environment. In this work, we propose a method, which includes realworld images alongside visual domain randomization in the reinforcement learning training procedure to further enhance the performance after sim-to-real transfer. We train variational autoencoders using both real and simulated frames, and the representations produced by the encoders are then used to train reinforcement learning agents. The proposed method is evaluated against a variety of baselines, including direct and indirect visual domain randomization, end-to-end reinforcement learning, and supervised and unsupervised state representation learning. By controlling a differential drive vehicle using only camera images, the method is tested in the Duckietown self-driving car environment. We demonstrate through our experimental results that our method improves learnt representation effectiveness and robustness by achieving the best performance of all tested methods.

Optimization of a line detection algorithm for autonomous vehicles on a RISC-V with accelerator

In recent years, autonomous vehicles have attracted theattention of many research groups, both in academiaand business, including researchers from leading com-panies such as Google, Uber and Tesla. This type ofvehicles are equipped with systems that are subjectto very strict requirements, essentially aimed at per-forming safe operations –both for potential passengersand pedestrians– as well as carrying out the process-ing needed for decision making in real time. In manyinstances, general-purpose processors alone cannotensure that these safety, reliability and real-time re-quirements are met, so it is common to implementheterogeneous systems by including accelerators. Thispaper explores the acceleration of a line detection ap-plication in the autonomous car environment using aheterogeneous system consisting of a general-purposeRISC-V core and a domain-specific accelerator. In par-ticular, the application is analyzed to identify the mostcomputationally intensive parts of the code and it isadapted accordingly for more efficient processing. Fur-thermore, the code is executed on the aforementionedhardware platform to verify that the execution effec-tively meets the existing requirements in autonomousvehicles, experiencing a 3.7x speedup with respect torunning without accelerator.

Modulation of Driver's Emotional States by Manipulating In-Vehicle Environment: Validation With Biosignals Recorded in An Actual Car Environment

A driver's emotional state can affect driving performance. According to the studies on the driving performance based on the circumplex (arousal-valence) model of affect, negative emotions such as anger and sadness can severely hinder safe driving. In this study, we developed a system to modulate drivers’ emotions to designated emotional states by manipulating in-vehicle environments, such as ambient lighting, background music, scent, ventilation, and rear curtains. The proposed system, named the “mood-modulator” system, consists of four different modes, designed to induce different emotional states. The feasibility of the “mood-modulator” system was evaluated using electroencephalogram (EEG) and photoplethysmogram (PPG) signals recorded from 48 drivers in an actual car environment. In the experiments, negative emotions were induced for each participant using short movie clips. Then, one of the four modes (different in-vehicle environments) was executed, during which both EEG and PPG data were acquired. We quantitatively evaluated whether each mode could effectively induce targeted emotional valence using machine learning classifier models, individually constructed from EEG data recorded during calibration sessions. The modulation of emotional arousal by each mode was also assessed using heart rate and respiration rate extracted from the PPG data. Our results demonstrated that the four modes could effectively increase the participant's emotional valence and modulate emotional arousal state to the intended direction. To the best of our knowledge, this is the first study to quantitatively evaluate a system that modulates a driver's emotional state using biosignals recorded in an actual car.

Effect of listener head orientation on speech reception threshold in an automotive environment

The head orientation of the listener significantly affects speech intelligibility (SI) in automobiles due to the effect of binaural listening and special acoustic conditions such as early reflections and seat-back occlusions. However, this issue has not been studied with subjective tests yet. This study investigates SI with various head orientations of a listener in automobiles with an subjective experiment. The sentence speech reception thresholds (SRT) in Mandarin Chinese are measured via headphones virtually in an automobile environment, and compared with results in a weak-reflective listening room. A virtual speaker is located in the front-passenger seat, the right-back seat and the left-back seat in sequence, by convolving the target speech with corresponding binaural room impulse responses measured on a dummy head in the driver’s seat with five head orientations. Result shows that the SRT variations caused by head orientations are up to 5 dB in automobile, lower than that of 9 dB in listening room. Under various head orientations, the SRT in automobile decreases as the virtual speaker moves closer to the front lateral direction, which is similar to the change rule in listening room that is mainly determined by the effect of binaural listening. Overall, a lower SRT in automobile can be obtained when the listener in the driver seat turns their head inward, i.e., towards the right. In comparison with the result in listening room, early reflections improve SI in automobile by an SRT decrease of up to 3.5 dB, while seat-back occlusions reduce SI by an SRT increase of up to 5.3 dB. The early reflections play a more significant role when the listener is in an adverse position, e.g., for the head orientations and speaker locations making direct sounds difficult to reach the listener’s ears. Comparison between the SRTs based on speech transmission index (STI) and STI-SI models and the SRTs measured by subjective experiment indicates that the STI-based objective method only partially expresses the variation in SI obtained from subjective experiment, and the difference between the subjective results and the objective results is mainly caused by the inapplicability of the STI-SI models derived from the traditional room in the automotive environment. The present work is relevant to understanding the combined effect of various factors on SI under such a special acoustic condition in automobile.

BloSM: Blockchain-Based Service Migration for Connected Cars in Embedded Edge Environment

Edge computing represents the future of computing paradigms that perform tasks near the user plane. The integration of blockchain with edge computing provides the added advantage of secured and trusted communication. In this paper, we propose a blockchain-based service migration by developing edge clusters using NVIDIA Jetson boards in an embedded edge environment, using containers and Kubernetes as a container orchestration capable of handling real-time computation-intensive deep learning tasks. Resource constraints in the edge and client movement are the proposed scenarios for service migration. Container migration due to mobile clients is integrated with blockchain to find a suitable destination, meta-based node evaluation, and secured data transfer in the connected car environment. Each service request migration takes, on average, 361 ms. The employed container migration method takes 75.11 s and 70.46 s to migrate application containers that use NVIDIA CUDA Toolkit. Finally, we evaluate the efficiency of blockchain to find the destination node through performance parameters such as latency, throughput, storage, and bandwidth.

Identification System Based on Resolution Adjusted 2D Spectrogram of Driver’s ECG for Intelligent Vehicle

Recently, traditional vehicles are being developed into intelligent vehicles as information is exchanged among various devices inside and outside the vehicles. In the connected car environment, the need for vehicle security is growing due to vehicle hacking accidents and possible threats to human life. Driver identification technology using electrocardiogram (ECG) signals has been studied to address vehicle security issues and driver-specific services. Existing driver identification systems tried to address the issues using a multidimensional feature extraction method. However, there are remaining issues, including accuracy concerns, because the resolution was adjusted without considering the ECG’s P, QRS Complexes, and T waves feature when analyzing the time-frequency multidimensional features. In this paper, we propose a driver identification system using a 2D spectrogram. It identifies a section where the resolution is optimally adjusted using a spectrogram that can simultaneously analyze the time-frequency features of an ECG. The experimental results show that the proposed method improved the identification performance compared to the existing multidimensional feature extraction methods such as EEMD and MFCCs. Besides, with a 2D spectrogram of 1/4 image size, the recognition performance is maintained in a CNN network and the training time is significantly reduced.

Deep Neuro-Vision Embedded Architecture for Safety Assessment in Perceptive Advanced Driver Assistance Systems: The Pedestrian Tracking System Use-Case.

In recent years, the automotive field has been changed by the accelerated rise of new technologies. Specifically, autonomous driving has revolutionized the car manufacturer's approach to design the advanced systems compliant to vehicle environments. As a result, there is a growing demand for the development of intelligent technology in order to make modern vehicles safer and smarter. The impact of such technologies has led to the development of the so-called Advanced Driver Assistance Systems (ADAS), suitable to maintain control of the vehicle in order to avoid potentially dangerous situations while driving. Several studies confirmed that an inadequate driver's physiological condition could compromise the ability to drive safely. For this reason, assessing the car driver's physiological status has become one of the primary targets of the automotive research and development. Although a large number of efforts has been made by researchers to design safety-assessment applications based on the detection of physiological signals, embedding them into a car environment represents a challenging task. These mentioned implications triggered the development of this study in which we proposed an innovative pipeline, that through a combined less invasive Neuro-Visual approach, is able to reconstruct the car driver's physiological status. Specifically, the proposed contribution refers to the sampling and processing of the driver PhotoPlethysmoGraphic (PPG) signal. A parallel enhanced low frame-rate motion magnification algorithm is used to reconstruct such features of the driver's PhotoPlethysmoGraphic (PPG) data when that signal is no longer available from the native embedded sensor platform. A parallel monitoring of the driver's blood pressure levels from the PPG signal as well as the driver's eyes dynamics completes the reconstruction of the driver's physiological status. The proposed pipeline has been tested in one of the major investigated automotive scenarios i.e., the detection and monitoring of pedestrians while driving (pedestrian tracking). The collected performance results confirmed the effectiveness of the proposed approach.

User Monitoring in Autonomous Driving System Using Gamified Task: A Case for VR/AR In-Car Gaming

Background: As Automated Driving Systems (ADS) technology gets assimilated into the market, the driver’s obligation will be changed to a supervisory role. A key point to consider is the driver’s engagement in the secondary task to maintain the driver/user in the control loop. This paper aims to monitor driver engagement with a game and identify any impacts the task has on hazard recognition. Methods: We designed a driving simulation using Unity3D and incorporated three tasks: No-task, AR-Video, and AR-Game tasks. The driver engaged in an AR object interception game while monitoring the road for threatening road scenarios. Results: There was a significant difference in the tasks (F(2,33) = 4.34, p = 0.0213), identifying the game-task as significant with respect to reaction time and ideal for the present investigation. Game scoring followed three profiles/phases: learning, saturation, and decline profile. From the profiles, it is possible to quantify/infer drivers’ engagement with the game task. Conclusion: The paper proposes alternative monitoring that has utility, i.e., entertaining the user. Further experiments with AR-Games focusing on the real-world car environment will be performed to confirm the performance following the recommendations derived from the current test.

Data Distribution for Multiple Receivers in a Connected Car Environment Using 5G Communication

The development of communication technology has brought changes to various environments. The evolution from 3G to 4G Long-Term Evolution (LTE) was mainly aimed at improving communication speed. However, the evolution from 4G LTE to 5G New Radio (NR) is not aimed at improving speed alone. In addition to the existing communication types, 5G aims to improve communication to support the Internet of Things (IoT), media, and complex content to which things are connected. In such environments, point-to-point communication has a very inefficient structure to allow content providers to transmit data to many content users. In the 5G era, content providers must distribute content to numerous users, and in this process, they need to protect the content. Multireceiver encryption (MRE) is an encryption technology developed for this purpose. MRE allows multiple recipients to decrypt data using their own private key with single encryption of a data provider. With this technology, even if the number of data recipients is 100,000 or 1,000,000, data can be distributed with single encryption. Therefore, while using the existing 1 : 1 encryption method, it is possible to solve the problem of inefficiency in performing encryption for each recipient. However, existing proposed MREs can cause key escrow problems and partial key verification problems. Furthermore, the privacy issues identifying the recipient may arise because anonymity is not available to the recipient. In addition, it is necessary to ensure a fair decryption process for all recipients which a legitimate user cannot decrypt. In this study, we attempted to address these problems, and through our model, it is possible to distribute the data more securely and efficiently in a 5G environment.

Convolutional Neural Networks(CNN) based Eye-Gaze Tracking System using Machine Learning Algorithm

To avoid the rising number of car crash deaths, which are mostly caused by drivers' inattentiveness, a paradigm shift is expected. The knowledge of a driver's look area may provide useful details about his or her point of attention. Cars with accurate and low-cost gaze classification systems can increase driver safety. When drivers shift their eyes without turning their heads to look at objects, the margin of error in gaze detection increases. For new consumer electronic applications such as driver tracking systems and novel user interfaces, accurate and effective eye gaze prediction is critical. Such systems must be able to run efficiently in difficult, unconstrained conditions while using reduced power and expense. A deep learning-based gaze estimation technique has been considered to solve this issue, with an emphasis on WSN based Convolutional Neural Networks (CNN) based system. The proposed study proposes the following architecture, which is focused on data science: The first is a novel neural network model that is programmed to manipulate any possible visual feature, such as the states of both eyes and head location, as well as many augmentations; the second is a data fusion approach that incorporates several gaze datasets. However, due to different factors such as environment light shifts, reflections on glasses surface, and motion and optical blurring of the captured eye signal, the accuracy of detecting and classifying the pupil centre and corneal reflection centre depends on a car environment. This work also includes pre-trained models, network structures, and datasets for designing and developing CNN-based deep learning models for Eye-Gaze Tracking and Classification.