Infrared colorization with cross-modality zero-shot learning

Abstract

Listen

Translate article

Existing infrared colorization methods based on image-to-image translation in a GAN framework require paired image data for network learning and are limited by data quality. They are also restricted in their colorization ability to perform spectral translation between infrared bands, specifically, long-wave infrared colorization methods cannot be directly applied to the near-infrared modality. This paper proposes a novel infrared colorization algorithm that leverages similarities in high-frequency features between visible and infrared images. The method achieves infrared colorization through frequency domain feature decoupling and reconstruction. By decoupling high-frequency and low-frequency features in the frequency domain, it retains similar high-frequency features and reconstructs removed low-frequency features. The network is trained using visible images, enabling cross-modality zero-shot learning, eliminating the need for infrared datasets during training. During inference, the low-frequency information of the infrared image’s frequency domain is removed, and the visible low-frequency information is supplemented using the trained network’s reconstruction capability, resulting in the desired coloring effect. The method is equipped with multi-modality cross-spectral colorization capability and performs well on multiple infrared spectral colorization tasks. The experimental results fully demonstrate the excellent cross-modality adaptability and broad spectral colorization capability of the method.