Exploiting high-fidelity kinematic information from port surveillance videos via a YOLO-based framework

Abstract

Port surveillance videos provide rich and intuitive temporal and spatial information and motion information, which is conducive to path planning, obstacle avoidance and subsequent risk prediction of automated guided vehicle (AGV) in automated container terminal (ACT). Extracting high-fidelity kinematic traffic relevant data from port surveillance videos become an active yet important research task in the ACT community. To that aim, the study proposes a deep-learning based framework to exploit spatial-temporal information from port videos with steps of object detection, object moving displacement mapping and speed estimation. First, the proposed framework detects both AGVs and people in the video with you only look once (YOLO) based model (i.e., YOLO V5 model). Second, we map the AGV (and people) moving distance from videos into physical world with the help of pinhole imaging rule. Third, we estimate AGV and people moving speed in the video with the help of linear regression model. The experiment results suggested that the proposed framework obtains satisfied performance considering that the average object moving displacement error is 0.19 m and the speed error is 0.08 m/s. The research findings help ACT management departments and regulations with high-fidelity traffic relevant data to further port safety and management efficiency.