Window-based transformer generative adversarial network for autonomous underwater image enhancement

Abstract

The estimation of high-quality underwater images is an important step towards the development of computer vision systems in marine environments. This fundamental step contains numerous computer vision and robotics applications including marine exploration, robotics manipulation, navigation, object detection, tracking, and sea life monitoring. However, this pre-processing step becomes more challenging in the presence of back-scattering of underwater particles and attenuation issues, which lead to the formation of hazy underwater images. Vision transformers have recently demonstrated outstanding performance in many computer vision applications. Window-based Transformers (WT) show promising enhancement performance by computing self-attention within non-overlapping local windows. WT has been identified as an essential component in improving representation capabilities; however, it has received less attention in improving the performance of Underwater Image Enhancement (UIE). Therefore, we propose a novel end-to-end Underwater window-based Transformer Generative Adversarial Network (UwTGAN). Our proposed algorithm consists of two main components, including a transformer generator that generates a restored underwater image and a transformer discriminator that classifies the generated underwater image. Both components are equipped with Window-based Self-Attention Blocks (WSABs), which maximize efficiency by limiting self-attention computation to non-overlapping local windows and provide relatively low computational costs. WSAB-based transformer generators and discriminators are trained end-to-end. We formulated an efficient loss function to ensure that the variables are closely integrated. Extensive experimental evaluations are performed on four independent underwater image datasets. Our results demonstrate that the proposed UwTGAN algorithm has outperformed several state-of-the-art UIE methods in terms of both quantitative and qualitative metrics by a significant margin.