HY-LSTM: A new time series deep learning architecture for estimation of pedestrian time to cross in advanced driver assistance system

Abstract

Advanced driver assistance systems (ADASs), particularly pedestrian protection systems (PPSs), have emerged as a hot research topic with the goal of enhancing traffic safety. The development of reliable on-board pedestrian detection systems is a critical problem for PPSs. It is extremely difficult to provide the required resilience of this type of system due to the fluctuating look of pedestrians (e.g., varied attire, changing size, aspect ratio, and dynamic shape) and the unstructured surroundings. The detection of pedestrians has gained huge focus among researchers because of its huge applications in the domain of automated vehicles. In the previous decades, the majority of examinations are done to obtain better solutions for detecting pedestrians, but fewer of them concentrated on determining the pedestrian. It is one of crucial interest to study regarding transport safety as it implies minimizing the count of traffic collisions and protecting pedestrians who are more susceptible to accidents. Predicting pedestrian conduct is critical for road safety, traffic management systems, ADAS, and autonomous vehicles in general. The fundamental problem in the field of self-driving and smart automobiles is identifying impediments, particularly people, and taking action to avoid collisions with them. Various studies have been conducted in this sector by many researchers, yet there are still many mistakes in the proper identification of pedestrians. Hence, this paper devises an approach to estimate pedestrian time for crossing in an advanced driving assistance system (ADAS). The inputted videos undergo keyframe extraction wherein crucial keyframes are extracted. The deep joint segmentation is further applied for identifying the pedestrian followed by intention classification. Then, the estimation of pedestrian time to cross is predicted using Hybrid-Long short term memory (HY-LSTM), and it is a new time series model obtained by unifying LSTM and Object-based convolution neural network (OCNN), where the layer and hyperparameters of OCNN are optimally derived using Gradient Chef Based Optimization (GCBO). The proposed GCBO-HY-LSTM outperformed showing the least Mean absolute error (MAE) of 0.038, Mean square error (MSE) of 0.029, and Root Mean square error (RMSE) of 0.170.