A Novel Multispectral Vessel Recognition Based on RGB-to-Thermal Image Translation

Abstract

Listen

Translate article

In the last decade, advances in deep learning have led to considerable progress in the field of ship classification in Red Green Blue (RGB) and Infra-Red (IR) images. However, ship classification performs poorly on images acquired in weak visible light intensity. Multispectral imaging constitutes a potential solution to address such difficulty. In this paper, we first propose Convolutional Neural Network (CNN) for ship classification in multi-spectral images (RGB, IR, etc.). The proposed architectures were trained from scratch and fine-tuned to another pre-trained network. Validation was carried out on the publically available RGB-IR pairs ship dataset VAIS. Unfortunately, owing to the small size of the dataset, the obtained classification result was 59,09%, hence not satisfactory for most applications. We, therefore, proposed a new image data augmentation approach for the generation of IR ship images from RGB images. The generation process was carried out through an adaptation of a Generative Adversarial Network (GAN) network and a Pix2Pix model. In fact, VAIS dataset was kept aside for validation purposes and KAIST RGB-IR pairs dataset was used for the training of our translator. The augmented IR dataset yielded more than a 9% increase in the performance of VAIS IR-based ship classification.