Real-time 360 degrees view for the operator of milliAmpere 2

Abstract

In the evolving domain of autonomous marine operations, accurate perception and representation of the surrounding environment are crucial for safe and effective execution. This paper addresses this issue by developing and testing a near real-time 360-degree bird’s eye view system for the situations where the ferry, milliAmpere 2, has to be manually controlled by a local operator onboard. The goal was to aid the operator during the critical phase of docking, by displaying the surrounding area of the ferry from a bird’s eye view. The bird’s eye view was made by using inverse perspective mapping on the undistorted images from the 8 cameras onboard. The system was implemented in Python, and aimed to reach a run-time of less than 200ms. This goal was reached during the initial phase of the work. However, during live testing, only near real-time performance was achieved. Despite some shortcomings, the operators found the system to be a “useful additional assistance” during the docking process.