Accident risk prediction model based on attention-mechanism LSTM using modality convergence in multimodal

Abstract

Anytime, anywhere, anyone can collect not only personal information but also surrounding situation information from home or a vehicle on the move without being restricted by time and space. To this end, the need for various devices such as biosensors, wearable devices, and temperature sensors is increasing. In this circumstance, there are many problems with the interaction of multiple devices and data processes, and there is a lack of structured or unstructured emerging contextual data. Accordingly, it is necessary to come up with a data collection method to obtain a variety of information efficiently. Such a method can obtain a diversity of information consistently with the use of data convergence. Since the method makes possible decision making, entity identification, and context prediction, it is applicable to diverse areas, including object detection and context awareness. The data generated through convergence makes it possible to the improve performance of data analysis through machine learning and deep learning. For this reason, a variety of modal types are applied to data analysis. Modal convergence for analysis can improve analysis performance more than unimodal. In this study, propose the accident risk prediction model based on attention-mechanism LSTM using modality convergence in multi-modal. In order to predict the risk of an accident, the proposed method makes the convergence of structured and unstructured modality data on the basis of judgmental fusion and statistical fusion and generates data sets. It is capable of solving the problem of data shortage and obtaining a variety of information consistently. In the proposed method, firstly, preprocessing is performed, and data sets of accident risk information are generated with the uses of the road risk equation and accident risk equation. Next, the correlation coefficient and regression coefficient for each variable is calculated through regression analysis modal and correlation coefficient analysis modal. A correlation coefficient is used to judge which variable needs to be learned intensively in order for the attention-mechanism of the accident risk prediction model. A regression coefficient is used as attention-weight to adjust a data weighting factor in attention-mechanism LSTM. Accordingly, whether there is the risk of an accident is judged. Performance evaluation is conducted in two ways. Firstly, epoch-based loss is evaluated in the comparison between the attention-mechanism LSTM-based accident risk prediction model with convergence modal and the model without it. In short, their RMSE is evaluated and compared. Secondly, the proposed model is compared with conventional models in terms of RMSE.