Pedestrian Motion Trajectory Prediction With Stereo-Based 3D Deep Pose Estimation and Trajectory Learning

Jianqi Zhong,Hao Sun,Zhihai He,Wenming Cao

doi:10.1109/access.2020.2969994

Abstract

Existing methods for pedestrian motion trajectory prediction are learning and predicting the trajectories in the 2D image space. In this work, we observe that it is much more efficient to learn and predict pedestrian trajectories in the 3D space since the human motion occurs in the 3D physical world and and their behavior patterns are better represented in the 3D space. To this end, we use a stereo camera system to detect and track the human pose with deep neural networks. During pose estimation, these twin deep neural networks satisfy the stereo consistence constraint. We adapt the existing SocialGAN method to perform pedestrian motion trajectory prediction from the 2D to the 3D space. Our extensive experimental results demonstrate that our proposed method significantly improves the pedestrian trajectory prediction performance, outperforming existing state-of-the-art methods.

Highlights

Human motion behaviors and trajectories are driven by human behavioral reasoning, common sense rules, social conventions, and interactions with others and the surrounding environment
We propose to extend the existing deep learning-based pose estimation and trajectory prediction from the 2D image space to the 3D space
Our experimentally results demonstrate that, by estimating, learning, and predicting the human pose and trajectories in the 3D space instead of the 2D image space, our method is able to significantly reduce the trajectory prediction error by up to 71% with an average error reduction of 47%

Summary

Introduction

Human motion behaviors and trajectories are driven by human behavioral reasoning, common sense rules, social conventions, and interactions with others and the surrounding environment. Human can effectively predict short-term body motion of others and respond . The ability for a machine to learn these rules and use them to understand and predict human motion in complex environments is highly valuable with a wide range of applications in social robots, intelligent systems, and smart environments [1], [2]. Predicting human motion is a very challenging task [3]. An efficient algorithm for human trajectory prediction needs to model physical constraints of the environment on human motion and anticipate movements of other persons or vehicles and their social behaviors. Earlier methods have been focused on learning dynamic patterns of moving agents

Objectives

Results

Conclusion