The Real-Time Human Reliability Detection System Based on Ship Bridge Videos

Abstract

Abstract Human reliability analysis in assessment of ship collision risk has always been a concern for shipping practitioners. The risk of ship collision is largely related to the driving state of crew. At the same time, although surveillance cameras have been deployed in key locations such as ship bridge on most of the current operating ships, the monitoring information is only used as evidence for daily evaluation or responsibility determination after an accident, further values are not discovered. How to make full use of existing ship bridge video resources to assess the realtime state of crew, and then assess the risk of collisions, is of great significance and academic value. Therefore, this paper proposes a real-time human reliability detection system based on ship bridge videos. Object detection model trained on COCO[1] dataset using Faster-RCNN [2] algorithm and spatio-temporal action detection model trained on AVA [3] dataset using SlowFast [4] algorithm are used to perform video understanding work on ship bridge videos, obtaining the actions of crew and corresponding probabilities. This paper introduces performance impact factors (PIFs) and Information, Decision and Action in a Crew content cognitive (IDAC) model to build a three-layer mapping model which clarifies how to use the results of video understanding work to calculate relevant PIF scores and explain the impact of the state of crew on various processes in IDAC model.