Reinforced Res-Unet transformer for underwater image enhancement

Abstract

Since light propagates through water with variable degrees of energy attenuation, the captured images suffer mainly from color deviations, lower contrast, and blurred details and textures. Compared with the traditional algorithm, the data-driven method has great advantages. However, the problems related to the reasonable network architecture, coding method and scarcity of databases still need to be explored to meet the requirements for high-quality reconstructed images in various tasks. In this paper, an underwater image enhancement network based on feature fusion is proposed, called RUTUIE. It combines the advantages of Resnet and the U-shape structure, mainly consists of an adaptation of multi-stage up-and-down sampling. Specifically, the U-shape structure serves as the backbone of Resnet, with two feature transformers on each side and a single level of up-and-down sampling. This structure aims to prevent insignificant features from being ignored during the process of the multi-stage up-and-down sampling. Moreover, the improved transformer encoder based on the feature-level attention mechanism combines the advantages of CNN, which allows the network to have both local and global perception. It can also consider the correlation between features in different subspaces during feature fusion. Then, we propose and demonstrate that embedding an adaptive feature selection module at appropriate locations can retain more learned feature representations. Moreover, we present how the previously proposed color transfer method can be utilized to synthesize underwater images and for network training. Extensive experiments demonstrate that our work effectively corrects color casts, reconstructs the rich texture information in natural scenes, and improves the contrast.