Towards automated identification of ice features for surface vessels using deep learning

Abstract

Ship traffic in ice exposed areas is increasing, and ice navigation is largely a manual task. Despite the progress in machine learning and computer vision algorithms, little focus has been given to computer-aided scene understanding in icy waters to help modernize navigational support systems. This work lays the foundation for the automated identification of ice for surface vessels using modern deep learning (DL) algorithms. The focus is on locating and classifying multiple ice objects within images from a surface vessel travelling through icy waters. The following categories of surface ice features are considered: level-ice, deformed ice, broken-ice, icebergs, floebergs, floebits, icefloes, pancake-ice, and brash-ice. In the first phase, we used DL algorithms to classify the ice objects in an image. For this task, seven state-of-the-art residual network (ResNet) models have been tested and include ResNet18, ResNet34, ResNet50, SE_ResNet50, Xception-Cadene, Inception-v4, and Inception-ResNet-v2. During the second phase, we used DL algorithms to locate and classify ice objects. For these tasks, we used the UNet architecture combined with conditional random fields (CRFs) and analysed the effects of using fully connected CRF and convolutional CRF. We have trained and validated the models using the close-range optical ice imagery, and then the promising models were used to classify and locate the different ice features in images from the bridge of the US Coast Guard icebreaker Healy and the nuclear-powered icebreaker 50 Let Pobedy. This paper provides the main findings and lessons that were learned from the execution of this study.