Driver information system: a combination of augmented reality, deep learning and vehicular Ad-hoc networks

Abstract

Improving traffic safety is one of the important goals of Intelligent Transportation Systems (ITS). In vehicle-based safety systems, it is more desirable to prevent an accident than to reduce severity of injuries. Critical traffic problems such as accidents and traffic congestion require the development of new transportation systems. Research in perceptual and human factors assessment is needed for relevant and correct display of this information for maximal road traffic safety as well as optimal driver comfort. One of the solutions to prevent accidents is to provide information on the surrounding environment of the driver. Augmented Reality Head-Up Display (AR-HUD) can facilitate a new form of dialogue between the vehicle and the driver; and enhance ITS by superimposing surrounding traffic information on the users view and keep drivers view on roads. In this paper, we propose a fast deep-learning-based object detection approaches for identifying and recognizing road obstacles types, as well as interpreting and predicting complex traffic situations. A single convolutional neural network predicts region of interest and class probabilities directly from full images in one evaluation. We also investigated potential costs and benefits of using dynamic conformal AR cues in improving driving safety. A new AR-HUD approach to create real-time interactive traffic animations was introduced in terms of types of obstacle, rules for placement and visibility, and projection of these on an in-vehicle display. The novelty of our approach is that both global and local context information are integrated into a unified framework to distinguish the ambiguous detection outcomes, enhance ITS by superimposing surrounding traffic information on the users view and keep drivers view on roads.