Multimodal Feature-Assisted Continuous Driver Behavior Analysis and Solving for Edge-Enabled Internet of Connected Vehicles Using Deep Learning

Omar Aboulola,Mashael Khayyat,Majid H Alsulami,Heba Fasihuddin,Ammar Muthanna,Mohammed Saleh Ali Muthanna,Basma Al-Harbi

doi:10.3390/app112110462

Abstract

The emerging technology of internet of connected vehicles (IoCV) introduced many new solutions for accident prevention and traffic safety by monitoring the behavior of drivers. In addition, monitoring drivers’ behavior to reduce accidents has attracted considerable attention from industry and academic researchers in recent years. However, there are still many issues that have not been addressed due to the lack of feature extraction. To this end, in this paper, we propose the multimodal driver analysis internet of connected vehicles (MODAL-IoCV) approach for analyzing drivers’ behavior using a deep learning method. This approach includes three consecutive phases. In the first phase, the hidden Markov model (HMM) is proposed to predict vehicle motion and lane changes. In the second phase, SqueezeNet is proposed to perform feature extraction from these classes. Lastly, in the final phase, tri-agent-based soft actor critic (TA-SAC) is proposed for recommendation and route planning, in which each driver is precisely handled by an edge node for personalized assistance. Finally, detailed experimental results prove that our proposed MODAL-IoCV method can achieve high performance in terms of latency, accuracy, false alarm rate, and motion prediction error compared to existing works.

Highlights

Namely, frequent change, no change, and mild change for improving the performance of the proposed MODAL-internet of connected vehicles (IoCV) method; RSU extracts multiple features from four classes, which are provided by SqueezeNet, which is a lightweight model. It extracts three types of features, including visual features, vehicular features, and smart IoT device features, which increases the speed and accuracy of feature extraction compared to AlexNet; We construct a perfect roadside virtual graph by considering vehicle mobility speed, moving direction, and the distance between vehicles for broadcasting the warning messages to the nearby vehicles and roadside pedestrians, which helps to reduce the risk of accidents; we provide a recommendation to the drivers by disseminating the message to each driver using edge nodes
A MODAL-IoCV method is proposed for driver behavior analysis using a deep learning approach. 5G is a high-speed and reliable technology, and the proposed model using edge computing possesses reduced latency due to the execution speed of the proposed SqueezeNet, which requires only a fewer number of parameters
The first layer includes the number of vehicles and RSU for monitoring the behaviors of the vehicles

Summary

Introduction

In intelligent transportation systems (ITSs), driver behavior analysis is the most exploited topic in terms of emergency events reduction [1]. If any abnormal behavior occurs on the roadside, the service highlights the wrong information to the driver, and affects the transportation system around the vehicle [2]. Accidents and road risks are caused by high volumes of traffic and the speed of vehicles [3]. Driver behavior analysis based on the continuous monitoring of visual features (driver’s actions) has been used for managing risks in the roadside environment [4,5]

Objectives

Findings

Discussion

Conclusion